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,  the  constraints  and  assumptions  that  underly  the  perception  of  surface  shape  and 
transparency.  The  identification  and  elucidation  of  these  assumptions  and  con¬ 
straints  is  important  for  the  design  of  flight  simulators,  in  order  that  syn¬ 
thetic  images  are  not  built  up  from  conflicting  ones  that  will  mislead  the  ob¬ 
server. 

The  first  part  of  the  report  examines  our  perception  of  surface  contours 
(e.g.,  the  projections  of  linear  surface  markings  such  as  seams,  wrinkles,  pig¬ 
mentation  edges,  glossy  reflections,  and  shadow  edges').  Generally  such  contours 
lie  interior  to  the  silhouette  of  an  object  and  have  been  regarded  as  merely 
contributing  to  texture,  or  to  making  the  surface  appear  solid,  or  simply  adding 
to  the  complexity  of  the  image.  In  fact,  they  constitute  an  important  source 
of  information  about  surface  shape  ^4  This  study  addresses  a  few  fundamental 
issues  concerning  our  perception  of/shape  from  contour:  the  information  content 
of  surface  contours  (remembering  that  we  usually  have  little  a  priori  knowledge 
that  we  may  bring  to  bear  in  their  interpretation),  the  computational  problems 
of  inferring  shape  from  contour,  and  the  sorts  of  constraints  that  would  be 
sufficient  for  solving  these  problems.  The  approach  taken  is  to  decompose  the 
problem  into  two  subproblems :  (1)  relating  the  surface  contour  in  the  image 

to  the  corresponding  curve  in  3-D  (called  the  contour  generator)  and  (2)  relating 
the  contour  generator  to  the  actual  surface  on  which  it  lies.  This  decomposition 
allows  one  to  focus  on  the  aspect  of  the  problem  governed  by  projective  geometry 
(the  relation  between  the  contour  generator  and  the  surface) . 

The  second  part  of  this  report  examines  the  problem  of  determining  trans¬ 
parency  from  visual  information.  Transparency  of  a  homogeneous  surface  is 
considered  as  a  limiting  case  of  a  screen  consisting  of  a  net  of  opaque  elements. 
When  the  elements  of  the  screen  become  invisible,  then  the  occluded  scene  is 
viewed  through  a  transparent  film.  The  basic  problem  of  transparency  is  to 
determine  which,  part  of  the  content  of  an  image  is  attributable  to  the  screen, 
and  which  part  arises  from  the  occluded  scene  behind  the  screen.  Although 
spatial  (and  temporal)  as  well  as  the  spectral  and  intensity  content  of  an 
image  may  be  used  for  such  judgements,  this  study  focuses  upon  the  color  and 
lightness  information  available  to  the  observer.  The  most  useful  cue  for  assess¬ 
ing  transparency  using  these  two  stimulus  dimensions  is  a  reduction  in  contrast 
or  chromatic  content.  This  reduction  can  be  determined  locally  from  image  in¬ 
formation  alone.  Although  such  a  solution  is  not  computational  optimal,  it  may 
be  very  practical  for  most  natural  scenes . 
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EFFICIENT  COMPUTATIONS  AND  REPRESENTATIONS 

OF  VISIBLE  SURFACES 

ABSTRACT 

When  the  visual  system  is  approached  as  performing  information  processing  tasks,  the  processor 
can  be  characterised  as  solving  a  computational  problem  [Marr,  1976;  Marr  &  Poggio,  1977].  The 
first  step  in  this  approach  is  to  formulate  the  given  problem  precisely.  This  report  addresses  two 
such  problems,  the  extraction  of  surface  shape  information  from  the  image  of  contours  lying  on  a 
surface  and  the  determination  of  transparency.  Our  preliminary  analyses  discuss  the  constraints 
and  assumptions  that  underly  the  perception  of  surface  shape  and  transparency.  The  identification 
and  elucidation  of  these  assumptions  and  constraints  is  important  for  the  design  of  flight  simulators, 
in  order  that  synthetic  images  are  not  built  up  from  conflicting  ones  that  will  mislead  the  observer. 

The  first  part  of  the  report  examines  our  perception  of  surface  contours  (e.g.,  the  projections  of 
linear  surface  markings  such  as  seams,  wrinkles,  pigmentation  edges,  glossy  reflections,  and  shadow 
edges).  Generally  such  contours  lie  interior  to  the  silhouette  of  an  object  and  have  been  regarded 
as  merely  contributing  to  texture,  or  to  making  the  surface  appear  solid,  or  simply  adding  to  the 
complexity  of  the  image.  In  fact,  they  constitute  an  important  source  of  information  about  surface 
shape.  This  study  addresses  a  few  fundamental  issues  concerning  our  perception  of  shape  from 
contour:  the  information  content  of  surface  contours  (remembering  that  we  usually  have  little  a 
priori  knowledge  that  we  may  bring  to  bear  in  their  interpretation),  the  computational  problems  of 
inferring  shape  from  contour,  and  the  sorts  of  constraints  that  would  be  sufficient  for  solving  these 
problems.  The  approach  taken  is  to  decompose  the  problem  into  two  subproblems:  (I)  relating  the 
surface  contour  in  the  image  to  the  corresponding  curve  In  3-D  (called  the  contour  generator)  and 
(2)  relating  the  contour  generator  to  the  actual  surface  on  which  it  lies.  This  decomposition  allows 
one  to  focus  on  the  aspect  of  the  problem  governed  by  projective  geometry  (the  relation  between  the 
image  surface  contour  and  its  contour  generator)  independently  from  that  governed  by  intrinsic 
geometry  (the  relation  between  the  contour  generator  and  the  surface). 

The  second  part  of  this  report  examines  the  problem  of  determining  transparency  from  visual 
information.  Transparency  of  a  homogeneous  surface  is  considered  as  a  limiting  case  of  a  screen 
consisting  of  a  net  of  opaque  elements.  When  the  elements  of  the  screen  become  invisible,  then  the 
occluded  scene  is  viewed  through  a  transparent  film.  The  basic  problem  of  transparency  is  to 
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determine  which  part  of  the  content  of  an  image  is  attributable  to  the  screen,  and  which  part  arises 
from  the  occluded  scene  behind  the  screen.  Although  spatial  (and  temporal)  as  well  as  the  spectral 
and  intensity  content  of  an  image  may  be  used  for  such  judgements,  this  study  focuses  upon  the 
color  and  lightness  information  available  to  the  observer.  The  most  useful  cue  for  assessing 
transparency  using  these  two  stimulus  dimensions  is  a  reduction  in  contrast  or  chromatic  content. 
This  reduction  can  be  determined  locally  from  image  information  alone.  Although  such  a  solution 
is  not  computationally  optimal,  it  may  be  very  practical  for  most  natural  scenes. 
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PART  I:  SURFACE  CONTOUR  ANALYSIS 
Kant  A.  Stevens 

1.  INTRODUCTION 

This  part  describes  geometrical  constraints  that  may  govern  the  way  in  which  we  perceive  surface 
shape  from  surface  contours  in  an  image.  In  figure  1,  for  example,  the  smooth  curves  are  seen  in 
3-D  as  lying  on  an  undulating  surface.  We  appreciate  not  only  the  shape  of  the  surface,  but  also  its 
spatial  orientation  relative  to  us,  and  to  some  extent  we  perceive  the  overall  surface  as  receding  in 
depth.  The  difficulty  we  face  in  interpreting  figure  I  as  merely  a  two-dimensional  family  of 
sinusoids  (which  it  is)  shows  that  we  impose  constraints  in  the  form  of  a  priori  assumptions.  Some 
of  these  assumptions  lead  us  to  interpret  certain  curves  in  the  image  as  being  surface  contours 
(which  correspond  to  actual  curves  across  3-D  surfaces);  others  constrain  the  inferred  surface  shape 
that  we  derive  by  analysis  of  the  surface  contours.  For  the  surface  percept  to  be  both  definite  and 
accurate,  such  constraints  must  define  a  unique  surface,  and  must  generally  be  valid. 

Although  many  have  considered  our  perception  of  the  shape  of  contours  (e.g.,  [Koffka,  1935, 
Rock.  1975]),  the  problem  of  how  surface  shape  can  be  correctly  inferred  from  the  observed  surface 
contours  has  received  virtually  no  attention.  The  primary  intentions  of  this  part  of  the  report  are 

(a)  to  formalize  the  computational  problem  of  recovering  3D  surface  shape 
from  contour, 

(b)  to  introduce  useful  and  valid  constraints  towards  its  solution,  and 

(c)  to  describe  why  those  constraints  are  useful. 

1.1  What  information  is  oarried  by  surfaoe  oontours? 

The  contours  In  figure  I  are  in  orthographic1  projection;  hence  we  cannot  derive  distance 

1.  Orthographic  projection  is  equivalent  to  ■  parallel  projection,  as  opposed  to  a  perspective  projection. 
Figure  1  demonstrates  that  we  may  perceive  shape  from  surface  contours  in  orthographic  projection.  Later 
we  will  see  that  assuming  that  the  projection  is  orthographic  (and  not  perspective  from  some  unknown 
viewing  geometry)  is  probably  necessary  in  the  analysis. 
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information  from  perspectivity  in  the  image.  But  the  shape  of  the  contours  does  provide  surface 
shape  information  in  two  forms.  In  the  vicinity  of  the  surface  contour  one  may  deduce  either: 

surface  orientation.  The  relative  surface  orientation  may  be  solved 
uniquely  (i.e.,  up  to  a  slant  reflection  since  the  projection  is  orthographic) 
or  only  to  within  a  restricted  range  of  slant  and  tilt. 

qualitative  surface  shape.  The  intrinsic  geometry  of  the  surface  may  be 
deduced  from  the  shape  of  the  surface  contours.  The  primitive  descriptors 
might  include  "flat",  "singly  curved",  "cylindrical",  "doubly  curved"  and  so 
forth.  This  sort  of  shape  information  is  independent  of  the  viewpoint. 

This  is  not  to  say  that  a  depth  map  may  not  be  computed  from  the  image,  but  that  the  geometry  of 
contours  in  an  orthographic  image  more  directly  constrains  surface  orientation  and  intrinsic 
geometry  than  distance  --  the  computation  of  a  depth  map  would  effectively  require  the 
intermediate  computation  of  surface  orientation. 

Note  that  information  about  intrinsic  surface  shape  serves  two  useful  purposes:  (a)  it  constitutes 
a  primitive,  coordinate-free  shape  descriptor,  and  (b)  it  constrains  the  values  in  any  representation 
of  surface  orientation  or  distance.  Suppose  that  it  can  be  determined  from  the  image  that  a  surface 
region  must  be  singly  curved,  then  this  restriction  can  be  imposed  on  any  independently  computed 
distance  or  surface  orientation  representation  --  the  distance  or  surface  orientation  must  vary  in  a 
manner  consistent  with  a  singly  curved  surface.  Later  we  shall  see  the  contribution  of  this 
qualitative  shape  constraint  on  the  computation  of  "shape  from  shading"  (c.f.,  [Horn,  1975)). 

1.2  Contours  and  contour  generators 

It  is  valuable  to  distinguish  between  a  contour  In  an  image  and  the  corresponding  curve  in  3-D, 
called  the  contour  generator,  that  projects  to  that  contour  (see  [Marr,  1976b]).  The  contour  generator 
is  a  physical  curve  which  lies  across  a  surface,  such  as  a  boundary  between  patches  of  differing 
reflectance  (e.g.,  a  pigmentation  marking),  a  discontinuity  in  illumination  (e.g.,  a  shadow  edge  cast 
across  the  surface)  or  a  discontinuity  in  surface  orientation  (e.g.,  a  crease).  The  contour  generator 
may  also  correspond  to  the  boundary  of  the  surface  from  the  given  viewpoint. 

So  on  the  one  hand,  we  have  the  contours  in  the  image;  on  the  other  hand,  their  corresponding 
physical  curves  in  3-D,  the  contour  generators.  To  make  3-D  interpretations  from  the  image 
contours  we  often  need  to  understand  what  causes  them  -  whether  they  correspond  to  object 
boundaries,  shadow  edges,  or  what. 
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Figure  2.  The  curves  in  a  are  interpreted  as  tangential  contours  and  the  underlying  surface  is  seen 
as  a  generalized  cone,  in  this  case,  a  vase-like  3D  shape.  Those  in  b  are  interpreted  as  surface 
contours  and  the  surface  appears  like  a  gently  curved  flag  or  a  ruled  sheet  of  paper. 
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One  basic  distinction  that  is  often  proposed  is  between  object  outlines  (also  termed  bounding 
contours  or  occluding  contours)  which  correspond  to  the  edge  of  an  object’s  silhouette  from  the  given 
viewpoint,  and  those  contours  that  lie  internal  to  the  silhouette  (which  Gibson  has  termed  "inlines"). 
A  slight  variant  would  be  to  distinguish  only  those  bounding  contours  that  correspond  to  the 
silhouettes  of  smooth  objects.  This  distinction  is  probably  fundamental  for  reasons  that  will  be  given 
in  the  following. 

1.3  Tangential  contours  and  surface  oontours 

Physical  objects  are  often  smooth,  and  their  silhouettes  alone  provide  a  strong  source  of 
information  about  the  overall  shape  [Marr,  1976b].  For  instance,  consider  a  vase.  Its  silhouette 
projected  onto  the  retinal  image  might  appear  like  the  upper  outline  shown  in  figure  2 a.  The 
contour  generator  in  3-D  of  this  outline  is  the  locus  of  points  along  which  the  line  of  sight  just 
grazes  the  surface  of  the  vase.  In  this  case,  the  contour  that  comprises  the  outline  will  be  termed  a 
tangential  contour.  An  important  class  of  such  contours  are  those  that  exhibit  qualitative  symmetry 
across  an  axis,  like  the  "vase"  in  Fig.  2a.  If  it  is  assumed  that  the  corresponding  surface  is  smooth 
then  its  silouette  is  that  of  a  generalized  cone  whose  3-D  shape  is  recoverable  (given  some  other 
restrictions,  see  [Marr,  1976b]).  In  this  case,  the  silhouette  boundary  is  comprised  of  tangential 
contours.  Note  that  the  surface  orientation  (its  slant  and  tilt)  is  known  at  all  points  along  a 
tangential  contour:  the  slant  is  ir/2  and  the  tilt  is  perpendicular  to  the  contour. 

In  the  previous  discussion  the  3D  object  was  assumed  to  have  a  smooth  surface.  However,  for 
objects  with  angular  faces  (such  as  many  man-made  objects),  or  objects  that  are  basically  2-D 
surfaces  (e.g.,  a  leaf),  the  surface  orientation  will  be  discontinuous  along  the  contour  generator 
which  corresponds  to  the  outline.  Since  the  line  of  sight  does  not  graze  the  surface  along  the  edge, 
the  silhouette  boundary  is  not  a  tangential  contour.  Observe  that  the  contours  in  figure  2b.  which 
we  interpret  as  the  outline  of  a  gently  curved  sheet,  present  a  fundamentally  different  problem  than 
the  contours  in  figure  2a.  Neither  do  we  assume  that  the  surface  is  smooth  nor  that  the  contours 
are  tangential  contours. 

The  distinction  that  I  propose  is  therefore  not  between  "outlines"  and  "inlines"  --  not  whether 
the  contour  is  along  the  boundary  of  the  silhouette  or  interior  to  the  bounary.  Instead,  the 
distinction  is  between  the  special  case  of  outline  contours,  the  tangential  contours,  and  all  other 
contours.  This  means  that  the  outlines  of  objects  that  are  not  smooth  will  be  treated  as  surface 
contours  for  our  purposes.  The  reason  for  this  is  the  following.  The  fact  that  a  given  contour  is 
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part  of  an  object  outline  does  not  constrain  the  shape  of  the  underlying  surface,  expect  when  the 
surface  is  smooth.  Otherwise,  the  contours  merely  delimit  the  visual  extent  of  a  object  from  the 
given  viewpoint.  The  rest  of  this  section  will  address  the  problem  of  using  surface  contours.  In 
general,  it  will  not  concern  us  whether  the  surface  contour  is  an  outline  contour  as  well. 

1.4  Surface  contours:  structural  and  illumination 

Thus  far,  we  have  only  distinguished  between  contours  corresponding  to  the  silhouettes  of  smooth 
objects,  the  tangential  contours,  and  all  other  contours  (those  being  collectively  termed  surface 
contours).  But  there  are  various,  distinct  physical  causes  of  these  surface  contours.  In  particular, 
we  can  distinguish  two  broad  categories  of  surface  contours,  roughly  speaking  by  whether  the 
associated  contour  generator  corresponds  to  a  physical  feature  on  the  surface  or  merely  due  to 
illumination.  The  first  category  will  be  termed  structural  contours,  the  latter,  Illumination  contours. 

Structural  contours  are  the  projections  of  contour  generators  which  mark  some  discontinuity  on 
the  surface,  e.g.  of  reflectance  or  of  surface  orientation.  Examples  that  occur  in  nature  are  given  by 
the  images  of  pigmentation  markings  on  a  zebra,  wrinkles  on  skin,  parallel  ridges  on  leaves,  rings 
on  bamboo  stalks,  and  cracks  on  wood  or  rock.  Images  of  synthetic  objects  commonly  present 
structural  contours  corresponding  to  seams,  sharp  edges,  groves,  and  pigmentation  markings. 

Illumination  contours  are  of  three  types:  (a)  the  projections  of  glossy  reflections,  such  as  those 
that  appear  on  metallic  or  wet  surfaces,  (b)  the  projections  of  shadow  edges  that  have  been  cast 
upon  a  surface,  and  (c)  the  images  of  self-shadows,  or  "terminators"  on  surfaces.  These  three  types 
have  been  grouped  together  as  illumination  contours  because  their  presence  is  strongly  dependent 
on  the  particular  illumination  and  may  shift  their  position  relative  to  the  surface  as  the  viewpoint 
or  light  source  geometry  changes.  They  are  all  potentially  useful  sources  of  information  about  the 
shape  of  the  surface,  as  we  shall  see,  but  since  they  depend  on  particular  arrangements  of 
illumination  and  viewing  geometry,  they  may  be  considered  as  fortuitous. 

We  now  turn  to  address  the  problem  of  constraining  the  visual  interpretation  of  surface 
contours.  First,  we  will  examine  a  decomposition  of  the  problem  into  two  steps,  each  of  which  must 
be  constrained.  Constraints  for  each  step  are  then  introduced,  and  their  validity  discussed. 
Discussion  of  how  these  constraints  are  computationally  useful  is  given  in  section  4. 
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2.  THE  CONSTRAINTS 

In  the  following  discussion  a  surface  will  be  denoted  by  E,  a  contour  generator  by  I*.  and  the 
projection  of  r  from  viewpoint  V  will  be  the  contour  Cy  (see  figure  3).  (When  the  viewpoint  is  not 
discussed,  the  contour  will  be  referred  to  simply  as  C.) 

A  surface  contour  in  the  image  is  the  projection*  of  a  contour  generator  T  lying  on  a  surface  £; 
neither  the  shape  of  I"  nor  £  is  known  a  priori.  Note  that  the  surface  contour  C  is  completely 
determined  by  the  3-D  locus  of  its  generator  T  in  space  relative  to  the  viewer,  regardless  of  the 
orientation  of  the  surface  on  which  T  lies  so  long  as  the  surface  allows  T  to  be  continuously  visible 
along  its  length.  This  is  an  important  point.  We  want  to  infer  the  shape  of  the  surface  £  from  the 
shape  of  the  surface  contour  C,  but  in  fact  C  is  not  a  function  of  the  shape  £;  C  is  only  a  function 
of  T.  In  orde.r  to  infer  the  shape  of  E,  the  relationship  between  T  and  £  must  be  constrained. 
Likewise,  to  infer  T  from  C,  the  relationship  between  I*  and  C  must  be  constrained.  The 
decomposition  that  is  suggested,  therefore,  involves  two  stages: 

(a)  inferring  the  shape  of  the  contour  generator  in  3-space  (C  =»  T)  then 

(b)  determining  how  the  surface  lies  under  the  contour  generator  (I*  =>  £). 

This  can  be  thought  of  as  (a)  bending  a  -wire  in  3-space  so  that  it  appears  to  the  viewer  as  does  the 
contour  in  the  image,  then  (b)  gluing  a  ribbon  along  the  wire  to  represent  the  strip  of  surface  that 
lies  directly  under  the  contour  generator.  In  these  terms,  we  see  that  infinitely  many  bendings  are 
possible  that  would  appear  identical  from  the  given  viewpoint,  and  the  ribbon  may  twist  arbitrarily 
along  the  wire.  These  two  aspects  of  the  problem  are  distinct. 

This  characterization  applies  equally  to  the  problem  of  inferring  surface  shape  from  multiple 
surface  contours  {C,}  in  the  image,  such  as  those  in  figure  I.  The  geometrical  arrangement  of 
particularly  if  they  are  parallel,  may  constrain  both  stages  I  and  II  (section  4.2.2).  Note  that  the 
appearance  of  figure  I  may  lead  one  to  suspect  that  parallelism  uniquely  constrains  the  surface,  but 
the  image  is  in  orthographic  projection  and  significantly  different  surfaces  may  project  to  the  same 


I.  The  projection  is  assumed  orthographic,  i.e.,  the  contour  generator  I"  is  assumed  small  compared  to  its 
viewing  distance.  The  perspective  distortions  otherwise  induced  in  C  would  be  infeasible  to  differentiate 
from  those  induced  by  slight  distortions  in  I*  under  conditions  of  orthographic  projection.  Note  further 
that  the  informal  term  "image  plane"  will  be  used,  although  the  retinal  projection  is  more  closely 
approximated  by  spherical  projection. 


-  ■  'Vk_- 

I**  1*1  dll-lli  . 


Richards  &  Stevens 


- 13  - 


Surface  contours 


Figure  3.  The  orthographic  projection  of  contour  generator  T  from  viewpoint  V  is  Cv.  The  curve 
Cw  is  termed  an  occluding  contour  if  it  is  an  edge  of  the  silhouette  of  an  object  from  viewpoint  V. 
In  particular,  if  the  line  of  sight  just  grazes  the  surface  along  T  then  the  curve  Cv  is  also  a 
tangential  contour.  The  curve  C¥  is  termed  a  surface  contour  if  it  is  not  a  tangential  contour. 
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image  --  the  separation  in  depth  between  the  contour  generators  on  the  surface  is  not  restricted  * 
Thus  even  in  the  case  of  multiple  parallel  contours,  the  surface  interpretation  process  must  be 
constrained,  and  that  constraint  is  naturally  described  in  terms  of  the  above  two  stages. 

This  decomposition  provides  a  framework  for  applying  constraints  to  the  problem  of  inferring 
E  from  C.  The  constraints  necessary  for  stage  I  involve  projective  geometry,  for  the  problem  is 
naturally  one  of  "deprojecting"  from  the  image  curve  to  the  curve  in  space.  The  constraints 
necessary  for  stage  II  do  not  involve  projective  geometry  --  they  do  not  depend  on  the  particular 
viewpoint.  Rather  they  involve  intrinsic  geometry,  specifically  the  relationship  between  the  curve 
on  the  surface  and  the  surface  itself. 

2.1  Some  geometrical  oonoepts 

This  section  reviews  some  concepts  that  are  necessary  for  discussing  the  relation  between  a  curve  on 
a  surface  and  the  underlying  surface  itself.  I  shall  review  the  notions  of  Gaussian  curvature,  lines 
of  curvature,  developable  surfaces  and  cylinders,  asymptotic  curves,  and  geodesics  (c.f.  [Hilbert  & 
Cohn-Vossen,  1952]). 

To  introduce  Gaussian  curvature,  consider  the  family  of  normal  sections  at  some  point  of  a 
smooth  surface,  i.e.,  the  contours  that  result  from  sections  that  contain  the  surface  normal  at  that 
point.  The  various  section  contours  through  that  point  usually  vary  in  curvature,  with  greatest  and 
least  curvature  occurring  at  two  principal  directions.  For  example,  contours  parallel  to  the  axis  of  a 
cylinder  will  be  lines  of  least  curvature  (they,  are  straight),  whereas  circles  in  planes  perpendicular  to 
the  cylinder’s  axis  will  be  lines  of  greatest  Gaussian  curvature.  (For  a  sphere,  the  curvature  is 
constant  everywhere  and  hence  this  shape  has  no  unique  principal  direction  of  curvature.)An 
important  property  of  the  two  principal  directions  is  that  they  are  mutually  orthogonal  at  every 
point  on  the  smooth  surface. 

The  Gaussian  curvature  at  a  point  is  the  product  of  the  greatest  and  least  curvatures.  The 
Gaussian  curvature  may  be  positive,  negative,  or  zero,  and  for  an  arbitrary  surface  may  vary 
continuously  across  the  surface.  For  example,  the  curvature  is  positive  on  a  smooth  pebble, 
negative  on  a  saddle  surface,  and  zero  on  a  cylinder  (defined  momentarily). 

1.  In  fact,  one  consistent  surface  solution  is  given  immediately  by  the  sheet  of  paper  on  which  figure  I  of 
this  article  is  printed  --  the  parallel  contour  generators  would  be  the  ink  on  the  page. 
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A  line  of  greatest  (or  least)  curvature  is  a  curve  whose  tangent  everywhere  coincides  with  one  of 
the  two  principal  directions.  Important  examples  are  the  cross  sections  and  meridians  of  surfaces  of 
revolution  (which  of  these  is  the  line  of  greatest  curvature  depends  on  the  surface  shape). 

A  developable  surface  is  a  surface  with  zero  Gaussian  curvature  everywhere  (i.e.,  the  curvature 
in  at  least  one  of  the  principal  directions  vanishes).  Thus  the  lines  of  least  curvature  are  straight 
lines  on  a  developable  surface.  Examples  of  developable  surfaces  are  planes,  cylinders,  and 
helicoids.  Informally,  they  correspond  to  the  class  of  surfaces  that  may  be  made  by  twisting  and 
curling  a  sheet  of  paper. 

A  cylinder  is  a  developable  surface  where  the  lines  of  least  curvature  are  parallel.  Cylinders 
may  be  formed  by  curling  a  sheet  without  torsion  -  it  may  be  rolled  into  a  tube  or  be  rippled  like  a 
hanging  curtain.  It  is  useful  to  think  of  a  cylinder  as  a  one-dimensional  surface. 

An  asymptotic  curve  is  a  locus  of  points  on  the  surface  where  the  Gaussian  curvature  is  zero. 
By  definition,  all  curves  on  developable  surfaces  are  asymptotic.  On  the  other  hand,  surfaces  with 
everywhere  positive  Gaussian  curvature  (such  as  a  sphere)  have  no  asymptotic  curves.  And 
surfaces  of  negative  Gaussian  curvature  must  have  asymptotic  curves,  since  the  principle  curvatures 
are  of  opposite  sign  and  for  some  direction  between  the  principle  directions  at  each  point  on  the 
surface  the  curvature  must  vanish. 

Finally,  a  geodesic,  usually  defined  as  the  shortest  path  between  two  points  on  a  surface,  is  also  a 
curve  whose  principal  normal*  everywhere  coincides  with  the  surface  normal.  Importantly,  the  lines 
of  greatest  and  least  curvature  on  a  cylinder  are  geodesics. 

2.2  What  constraints  might  bs  useful? 

We  now  introduce  some  constraints  that  allow  solutions  to  steps  I  and  II.  They  are  provided  by 
restricting  the  geometrical  properties  of  the  contour  generators,  and  restricting  the  relationship 
between  the  contour  generators  and  the  surface  on  which  they  lie.  This  section  only  tabulates  the 
various  geometric  restrictions.  Next,  in  section  3  we  will  discuss  the  validity  of  assuming  that  these 
restrictions  hold  in  natural  situations  involving  actual  contour  generators  on  physical  surfaces  and. 


I.  The  principal  normal  to  a  planar  curve  it  the  perpendicular  to  the  tangent  to  the  curve  and  liet  in  the 
plane  of  the  curve.  The  principal  normal  to  a  curve  with  torsion,  similarly,  is  perpendicular  to  the  tangent 
but  lies  in  the  osculating  plane  of  the  curve  at  that  point  (where  the  osculating  plane  is  defined  by  two 
successive  tangents  at  the  given  point).  Note  that  we  will  often  restict  curves  to  be  planar,  so  filing  the 
plane  of  a  geodesic  immediately  fixes  the  normal  to  the  surface. 
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in  section  4,  we  will  describe  how  the  restrictions  constrain  the  shape-from-contour  analysis. 

2.2.1  Constraints  on  the  contour  generator 

With  regard  to  step  I,  the  3-D  shape  of  a  contour  generator  T  (corresponding  to  a  given  surface 
contour  C)  may  be  recovered  if  restrictions  are  imposed  on  T  and  on  the  viewing  position.  Some  of 
these  restrictions  are  listed  below. 


(a)  general  position,  the  viewpoint  is  not  misleading.  This  allows  one  to 
infer  properties  of  the  contour  generator  T  on  the  basis  of  the  properties  of 
its  image,  the  surface  contour  C.  For  instance,  if  C  is  smooth  then  T  is 
smooth;  if  {Cjl  are  parallel  then  {1^}  are  parallel. 

(b)  planarity,  T  is  planar.  This  reduces  the  problem  of  determining  T  to 
that  of  determining  the  orientation  of  the  plane  n  containing  T.  The 
plane  n  is  constrained  by  the  following. 

(c)  symmetry.  Given  planarity  and  general  position,  if  C  presents  evidence 
of  symmetry  then  T  is  symmetric,  and  the  orientation  of  II  must  be 
consistent  with  T  being  symmetric. 

(d)  minimum  curvature  variation.  Given  planarity  and  general  position,  if 
the  curvature  of  T  is  roughly  constant  then  the  variations  in  curvature 
apparent  in  C  may  be  attributed  to  foreshortening.  Consequently  that 
plane  fl  that  minimizes  the  variation  in  curvature  of  I*  would  solve  T. 


2.2.2  Constraints  on  the  relation  between  contour  generator  and  surface 

Given  the  contour  generator  T,  the  surface  E  may  be  solved  if  the  relationship  between  I"  and  E  is 
restricted.  If  T  is  planar  and  lies  on  some  plane  n  then  the  relationship  between  the  contour 
generator  and  the  surface  is  naturally  described  in  terms  of  the  angle  between  FI  and  the  tangent 
plane  to  E  for  points  along  I*.  The  relation  between  the  surface  and  the  contour  generator  is  quite 
simple  if  we  make  the  strong  restriction  that  this  angle  is  constant  along  the  length  of  T.  That  is  to 
say,  the  plane  containing  the  contour  generator  meets  the  surface  at  a  constant  angle.  The  two 
cases  we  will  consider  is  when  the  angle  is  it  12  and  zero. 

If  the  angle  between  II  and  the  tangent  plane  to  E  is  ir/2,  then: 

T  is  geodesic.  The  surface  normal  coincides  with  the  principal  normal  to  T 
for  points  along  T. 

If  the  angle  between  ft  and  the  tangent  plane  to  E  is  zero,  then: 
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T  is  asymptotic.  The  surface  normal  coincides  with  the  normal  to  n  for 
points  along  T,  and  furthermore,  the  Gaussian  curvature  of  E  for  points 
along  T  is  zero. 

These  two  solutions,  geodesic  and  asymptotic,  form  the  basis  for  constraining  the  relation  between 
the  contour  generator  and  the  surface.  Given  general  position  and  planarity,  we  also  have  an 
important  restriction  on  £  in  the  case  of  parallel  surface  contours  (C{): 

{I*,}  are  parallel  lines  of  curvature  and  E  is  a  cylinder.  Furthermore,  if  the 
contour  generators  are  geodesics,  they  are  lines  of  greatest  curvature;  if 
asymptotics,  the  surface  degenerates  to  be  planar. 

And  finally,  a  derivative  of  the  cylinder  restriction  may  apply  in  the  case  of  a  single  surface 
contour,  if  the  corresponding  contour  generator  is  a  line  of  greatest  curvature  and  the  surface  is 
cylindrical,  by  the  following  restriction: 

E  is  opaque.  The  image  of  an  individual  line  of  greatest  curvature  on  a 
cylinder  allows  some  restriction  on  the  shape  of  the  surface. 

Surface  contours  are  often  weak  sources  of  information  about  the  surface  shape  when  analyzed 
individually,  primarily  because  it  is  difficult  to  deduce  the  shape  of  the  contour  generators  on  an 
individual  basis.  The  more  important  case  probably  involves  the  geodesic  restriction  on  a 
collection  of  parallel  contours  taken  together.  Then  the  parallelism  may  be  used  to  advantage  in 
constraining  the  shape  of  both  the  contour  generators  and  the  surface  on  which  they  lie.  Before 
pursuing  the  utility  of  these  constraints  any  further,  it  is  important  to  gain  some  insight  into  their 
validity. 


*■•*•*•  -  •  ’  ...  jl. 
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3.  WHEN  ARE  THE  CONSTRAINTS  VALID? 

Do  the  contour  generators  in  the  real  world  meet  these  restrictions?  In  some  situations  it  is  valid  to 
assume  that  a  contour  generator  is,  say,  planar  and  geodesic,  as  we  shall  see.  But  there  are  also 
instances  where  the  same  assumptions  are  not  valid  -  the  real  world  does  not  necessarily  constrain 
the  curves  on  surfaces  to  comply  with  any  of  the  various  ideal  geometries.  How  often  are  the 
restrictions  met  in  actuality?  This  is  the  issue  of  "ecological  validity"  discussed  by  Gibson, 
Brunswick,  and  others  (c.f.  [Gibson,  1950;  Postman  Sc  Tolman,  1959]).  We  start  with  considering  the 
validity  of  assuming  general  position. 

3.1  General  position 

General  position  implies  that  the  viewpoint  is  representative  -  that  the  image  taken  from  this 
position  does  not  mislead  us  by  accidental  alignments.  Two  examples  of  viewpoints  that  are  not 
general  position  may  be  imagined  for  a  cube:  In  one  instance  the  cube  is  positioned  so  that  its 
silhouette  is  a  regular  hexigon.  Equally  misleading  would  be  a  cube  positioned  so  that  its 
silhouette  is  a  perfect  square. 

When  the  assumption  of  general  position  is  correct  we  may  make  valid  deductions,  in 
particular,  deductions  about  contour  generators.  Two  examples  of  these  deductions  which  we  shall 
pursue  are  the  following:  If  a  surface  contour  is  smooth,  the  corresponding  contour  generator  is 
smooth,  and  if  surface  contours  are  parallel,  their  contour  generators  are  also  parallel. 

The  contour  generator  need  not  be  smooth  simply  because  its  projection  is  smooth:  a 
discontinuity  in  tangent  along  a  contour  generator  might  be  hidden  from  the  given  viewpoint  -  the 
plane  containing  the  discontinuity  might  also  contain  the  line  of  sight  so  that  the  discontinuity 
would  not  be  apparent.  But  if  the  distribution  of  spatial  orientations  of  planes  relative  to  the 
viewer  is  uniform,  the  likelihood  of  such  an  accidental  alignment  would  be  insignificant.  Similarly, 
some  non-parallel  curves  may  be  constructed  such  that  they  appear  parallel  from  certain  viewpoints, 
but  the  probability  of  achieving  a  viewing  position  that  allows  this  alignment  becomes  insignificant 
as  the  curves  diverge  from  parallelism  in  3-space.  * 

1.  Implicit  in  the  above  argument  is  the  reasonable  expectation  that  the  instances  of  actual  parallelism, 
straightness,  and  so  forth,  are  more  probable  than  accidental  alignments. 
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3.2  Geometrical  properties  of  structural  contours 

In  general,  the  geometry  of  structural  contours  is  not  strongly  constrained  because  the  processes  that 
cause  them  are  varied  and  often  random.  There  are,  however,  some  types  of  physical  markings 
that  are  well  constrained. 

The  clearest  examples,  perhaps,  involve  synthetic  objects.  With  reference  to  the  objects  about 
you,  observe  that  the  smooth  surfaces  of  man-made  objects  are  usually  comprised  of  either  (a) 
planar  surfaces,  (b)  singly  curved  surfaces,  in  particular  cylinders,  or  (c)  surfaces  of  revolution.  In 
general,  the  boundaries  between  surfaces  are  planar,  primarily  for  reasons  of  fabrication.  Again, 
because  of  convenience  in  manufacturing  as  well  as  utility,  curved  surfaces  are  usually  sliced  by 
normal  sections.  Thus  joints  between  surfaces  of  an  object  comprise  geodesics  on  one  or  the  other 
of  the  joining  surfaces.  The  end  of  a  "tin  can"  would  be  an  example.  Surface  markings  other  than 
seams  or  joints  are  often  geodesics  as  well,  particular  when  the  markings  are  on  cylinders.  When 
the  markings  are  also  planar,  they  additionally  constitute  lines  of  curvature.  This  combination  of 
properties,  planarity  and  geodesic,  is  particularly  common. 

Markings  on  surfaces  of  revolution  usually  follow  either  the  axis  or  some  cross  section.  Hence 
these  seams,  edges,  ridges,  and  pigmentation  markings  are  lines  of  curvature,  geodesic,  and  planar. 
(A  notable  exception  can  be  found  in  the  spiral  seams  on  cardboard  tubes.  They  are  geodesic  but 
nonplanar.) 

Flexible  surfaces,  both  natural  and  synthetic,  tend  to  be  noncompressible  hence  developable,  and 
are  therefore  cylinders  when  not  subjected  to  torsion.  Wrinkles  produced  by  compression  tend  to  be 
lines  of  curvature. 

Many  biological  forms  may  be  approximated  as  being  composed  of  generalized  cones  [Marr, 
1976bl  These  surfaces  often  have  markings  that  follow  cross  sections  and  meridians  on  the  surface, 
and  therefore  are  also  lines  of  curvature,  geodesic,  and  planar.  Biological  objects  are  often 
bilaterally  symmetric,  such  as  leaves.  Their  axes  of  symmetry  are  often  evidenced  by  physical 
markings,  and  symmetric  patterns  are  usually  arranged  across  that  axis.  The  symmetry  may  be 
used  to  advantage  to  restrict  the  possible  orientations  that  would  be  consistent  with  the  3-D  form 
being  symmetric. 
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3.3  Oeometrioal  properties  of  illumination  contours 
3.3.1  Cast  shadows 

The  edge  of  a  shadow  cast  across  a  surface  is  a  fortuitous  source  of  information  about  surface 
shape.  We  are  familiar  with  the  effectiveness  of  the  shadow  a  fence  post  cast  upon  snow  in 
indicating  the  undulations  in  the  surface.  But  to  accurately  analyze  the  surface  from  the  image  of 
the  cast  shadow,  a  number  of  variables  must  be  known.  There  are  essentially  two  projections 
involved:  the  projection  of  the  shadow  onto  the  surface  (the  edge  of  which  becomes  the  contour 
generator  T)  and  the  subsequent  projection  of  T  onto  the  image  plane  (as  contour  C).  Thus  the 
contour  C  in  the  image  depends  on  (a)  the  shape  of  the  physical  shadow-casting  edge,  (b)  the 
position  of  the  light  source  -  together  they  specify  the  bundle  of  rays  that  will  be  cast  upon  the 
surface  -  and  (c)  the  position  of  the  shadow-casting  edge  relative  to  the  surface,  and  finally  (d)  the 
shape  of  the  surface  itself. 

To  appreciate  the  complexity  of  shadow  interpretation  in  the  general  case,  consider  again  the 
image  of  a  tree  trunk  shadow  cast  on  snow.  Suppose  there  is  a  kink  along  the  shadow  edge.  Is 
that  due  to  a  sharp  depression  in  the  snow  (for  instance,  is  the  shadow  falling  across  a  footprint)  or 
is  it  due  to  a  kink  in  the  tree  (and  the  snow  itself  is  flat)?  If  analyzing  the  shape  of  the  surface  is 
attempted  prior  to  knowing  the  above  factors,  some  assumptions  are  necessary.  In  the  approach 
suggested  here,  the  assumptions  are  two: 

the  contour  generator  is  planar  and  gtodtsie. 

In  terms  of  this  example,  the  above  translate  into  assuming  the  edge  casting  the  shadow  is  straight 
and  that  its  profile  (determined  by  the  sun  position  and  the  trunk)  intersects  the  ground  at  a  right 
angle.  Then  if  there  is  an  apparent  kink  in  the  shadow  edge  it  will  be  attributed  to  the  surface,  not 
to  the  tree.  (Incidentally,  it  is  informative  to  observe  the  shadow  cast  on  the  flat  ground  by  a  young 
tree  which  has  a  crooked  trunk.  The  ground  often  appears  to  undulate  according  to  the  curves  in 
the  cast  shadow.) 

So  we  should  discuss  how  the  planarity  and  geodesic  restrictions  help  the  shape  analysis.  First 
note  that  if  the  shadow-casting  edge  is  straight  the  contour  generator  (the  shadow  edge  cast  across 
the  surface)  constitutes  a  planar  section  of  that  surface.  That  is,  the  contour  generator  lies  in  the 
plane  defined  by  the  straight  shadow-casting  edge  and  the  point  light  source.  In  this  case,  we  may 
already  determine  qualitative  information  about  the  surface  shape.  Given  general  position,  if  the 
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contour  in  the  image  corresponding  to  the  shadow  edge  is  straight,  the  surface  is  flat;  if  it  is 
curved,  the  surface  is  curved.  To  determine  more  quantitative  shape  information  requires  that  (a) 
the  relation  between  the  contour  generator  T  and  the  surface  be  known,  and  (b)  the  orientation  of 
the  plane  of  T  be  known.  Hence  we  introduce  the  geodesic  assumption.  That  is  to  say,  the  shadow 
edge  across  the  surface  is  assumed  to  be  a  normal  section  of  the  surface.  Weak  justification  for  this 
assumption  derives  from  considering  shadows  cast  on  the  ground-.  Since  shadow-casting  edges  are 
usually  vertical  (e.g„  tree  trunks,  building  edges,  telephone  poles,  fences),  the  edge  of  the  shadow 
amounts  to  a  normal  section,  i.e.,  the  shadow  edge  is  roughly  geodesic. 

When  do  multiple,  parallel  sections  occur  in  real  situations?  We  may  disregard  the  shadow  of  a 
picket  fence  as  being  artificial,  but  notice  that  two  parallel  sections  would  result  from  the  shadow 
edges  cast  on  some  surface  by  a  relatively  narrow  object  such  as  a  tree  trunk.  Another  possibility 
concerns  motion:  successive  views  of  a  moving  shadow  edge.  Successive  positions  of  a  shadow 
edge  that  sweeps  across  a  surface  in  translatory  motion  would  constitute  parallel  sections  of  the 
surface.  Does  the  visual  system  take  advantage  of  this  fact?  Is  our  ability  to  analyte  parallel 
surface  contours  a  derivative  of  an  ability  to  analyze  moving  shadows?  This  hypothesis  would  be 
supported  if  we  could  perceive  a  surface  defined  only  by  a  sing  it  moving  contour  that  scans  across 
an  otherwise  invisible  surface.  In  fact,  this  ability  may  be  demonstrated  by  a  motion  sequence  of  a 
single  contour  on  a  CRT,  where  each  frame  presents  only  a  single  curve.  Note  that  the  moving 
curve  might  be  interpreted  simply  as  a  flexible  wire  that  bends  as  it  translates,  or  more  literally,  as 
a  curve  in  the  plane  of  the  screen  that  changes  shape  as  it  moves.  But,  in  fact,  there  are  instances 
when  we  interpret  the  moving  contour  as  a  shadow  edge  sweeping  across  a  3-D  surface  (e.g.,  when 
the  individual  curves  in  figure  I  are  presented  in  succession). 

SJ.2  Specular  reflections:  gloss  contours  and  highlights 

Gloss  contours,  like  shadows,  are  fortuitous,  i.e.,  useful  but  not  necessarily  present.  They  are  present 
only  under  directional  lighting  conditions  on  specular  surfaces,  when  the  surface  normal  lies  in  the 
plane  defined  by  the  point  light  source,  surface  point,  and  viewer  and  bisects  the  angle  defined  by 
that  configuration.  This  configuration  (the  specularity  condition)  is  rarely  met  with  planar  surfaces 
but  is  commonplace  for  curved  surfaces,  especially  when  viewed  indoors  with  multiple  lights 
illuminating  the  surface.  The  specularity  condition  may  be  met  only  at  an  isolated  point,  causing  a 
highlight,  or  met  along  a  curve,  causing  a  gloss  contour. 

For  a  doubly  curved  patch  of  surface  the  specularity  condition  is  met  at  only  a  point,  if  at  all, 
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and  would  only  produce  a  highlight  in  the  image.  A  gloss  contour  cannot  occur  on  a  surface  with 
nonzero  Gaussian  curvature  in  orthographic  projection  given  a  point  light  sourced  For  a  gloss 
contour  to  occur  --  for  the  specularity  to  appear  not  as  a  point  but  as  a  curve  -  the  specularity 
condition  must  be  met  a>ong  a  continuous  curve  on  the  surface.  With  orthographic  projection  and 
distant  light  source  it  is  necessary  that  the  contour  generator  (the  locus  along  which  the  specularity 
condition  is  met)  be  planar.  That  plane  corresponds  to  the  tangent  plane  to  the  surface  along  the 
contour  generator.  Now  two  results  in  differential  geometry  are  useful: 

A  curve  is  asymptotic  if  it  lies  in  a  plane  everywhere  tangent  to  the  surface 
along  the  curve. 

If  the  angle  between  a  planar  curve  and  the  tangent  plane  of  the  surface  is 
constant,  then  that  curve  is  a  line  of  curvature. 


Using  the  above,  we  may  conclude  that  the  curve  across  the  surface  that  corresponds  to  the  gloss 
contour  is  asymptotic  and  a  line  of  (least)  curvature.  Since  the  asymptotic  curve  follows  a  path  of 
zero  Gaussian  curvature,  we  have  information  about  the  intrinsic  geometry  in  the  vicinity.  Of 
importance  is  the  following  which  hold  subject  to  general  position: 

If  the  gloss  contour  is  curved,  the  surface  is  planar.  This  is  true  in 
orthographic  projection  with  distant  light  source.  (With  nearby  objects 
and  perhaps  nearby  illumination,  the  surface  would  not  be  strictly  planar. 

But  in  general  the  surface  curvature  measured  along  the  contour  generator 
will  be  small,  much  less  than  that  measured  across  the  contour  generator.) 

If  the  gloss  contour  is  straight,  the  surface  is  cylindrical  when  either  (a) 
gloss  contours  from  successive  viewpoints  are  parallel,  or  (b)  if  there  are 
multiple  light  sources  (as  is  common  in  interior  scenes)  and  multiple  gloss 
contours  are  parallel. 

Thus  the  specular  reflections  in  the  image  can  tell  us  not  only  something  of  the  reflectance 


1.  In  real  initiation*  we  have  two  way*  in  which  gloss  contour*  may  arise.  First,  extended  light  sourer* 
(*uch  as  fluorescent  lights,  bright  window*)  will  extend  point  reflections  into  images  of  the  light  sources, 
which  appear  as  glows  contours  if  compressed  because  the  two  principle  curvatures  are  very  different. 
Secondly,  in  perspective  projection  we  may  have  that  as  the  line  of  sight  sweeps  across  the  surface  (the 

projection  is  not  parallel)  the  angle  between  the  line  of  sight  and  the  surface  stays  relatively  constant  due 

to  curvature  of  the  surface,  such  as  when  viewing  the  inside  surface  of  a  cup  from  nearby.  Then  if  the 

specularity  condition  is  met  at  one  point  in  that  vicinity,  it  would  be  met  along  a  locus.  Thu*  in 

perspective  projection  highlights  may  spread  into  gloss  contours  as  well. 
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properties  of  the  surface,  that  the  surface  is  specular  [Beck,  1972],  but  also  something  about  the 
surface  shape,  namely,  that  the  Gaussian  curvature  is  nonzero  in  the  vicinity  of  a  highlight  and 
zero  in  the  vicinity  of  a  gloss  contour.  The  shape  of  the  gloss  contour  also  specifies  the  intrinsic 
shape  of  the  developable  surfaced  This  does  not  strictly  hold  when  the  surfaces  or  light  sources  are 
near  by,  and  especially  when  the  light  comes  from  an  extended,  rather  than  a  point,  source. 
Nonetheless,  it  is  instructive  to  observe  the  gloss  contours  on  specular  surfaces  --  they  almost 
invariably  follow  the  least  curvature  paths  on  actual  surfaces. 

3.3.3  Shading  contours  and  terminators 

The  previous  discussion  assumes  bright,  directional  light  sources.  However  the  specular  surface  not 
only  reflects  the  light  sources  as  a  highlight  or  gloss  contour,  but  also  acts  as  a  mirror  --  the  various 
glossy  reflections  comprise  an  image  of  the  surrounds  distorted  by  the  geometry  of  the  surface. 
This  is  the  extreme  case  of  mutual  illumination  which  makes  ’shape  from  shading"  difficult.  The 
incident  illumination  is  an  intractably  complex  function  of  the  surrounds.  But  without 
understanding  this  illumination,  the  shape  of  the  surface  cannot  be  solved  from  the  shading. 

With  the  addition  of  a  matte  component,  the  fine  details  in  the  reflections  are  lost,  and  the  gloss 
contours  become  less  definite.  In  the  limit  case  of  a  Lambertian  surface  there  is  no  specular 
component  and  the  shading  is  only  a  function  of  the  surface  orientation  relative  to  the  various 
sources  of  illumination.  For  this  reason  one  would  expect  that  the  surface  orientation  would  be 
computed  from  shading  most  feasibly,  however  the  illumination  is  still  determined  by  the  surrounds 
and  is  still  quite  unconstrained.  Consequently,  the  computation  of  shape  from  shading  (where 
"shape"  means  local  surface  orientation)  is  quite  difficult. 

Most  surfaces  are  neither  totally  matte  nor  glossy  so  their  images  present  weak  highlights  and 
gloss  contours  --  the  distinction  between  shading  and  gloss  becomes  vague.  One  may  postulate, 
therefore,  that  shading  only  constrains  the  local  surface  geometry  in  the  manner  just  described  - 
the  local  surface  orientation  is  not  computed  directly  from  the  shading.  Instead,  the  local  surface 
orientation  would  be  smoothly  interpolated  between  those  tangential  contours  and  surface  contours 
along  which  surface  orientation  can  be  solved.  The  interpolation  would  be  subject  to  the  constraint 


1.  Furthermore,  the  surface  normal  coincides  with  the  normal  to  the  plane  containing  the  gloss  contour, 
but  to  utilize  that  fact  the  3-D  curve  corresponding  to  the  gloss  contour  must  be  determined.  That  is 
the  topic  of  section  4.1. 
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on  intrinsic  surface  geometry  provided  by  the  gloss  and  shading  contours.  This  constraint  is 
naturally  described  in  terms  of  Caussian  curvature:  A  highlight  indicates  positive  Gaussian 
curvature  in  the  vicinity.  Similarly,  a  gloss  contour  indicates  a  locus  of  zero  Gaussian  curvature. 

Constraint  on  intrinsic  geometry  is  also  provided  by  the  shading  contours  known  as  terminators, 
surface  contours  which  correspond  to  paths  on  the  surface  along  which  the  light  grazes  the  surface 
so  that  points  on  one  side  of  the  contour  are  illuminated,  points  on  the  other  side  are  in  shadow. 
(A  terminator  is  analogous  to  a  tangential  contour  seen  from  the  light  source  position.)  A  strong 
restriction  on  the  surface  shape  is  provided  wherever  the  terminator  is  straight  in  the  image:  the 
surface  is  locally  developable  (again,  assuming  general  position)  and  therefore  the  terminator 
indicates  a  locus  of  zero  Gaussian  curvature. 
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4.  HOW  THE  CONSTRAINTS  ARE  USEFUL 

Thus  far  we  have  discussed  a  number  of  geometrical  properties  that  may  be  useful  in  constraining 
the  analysis  of  shape  from  surface  contours.  Instances  in  which  these  properties  hold  in  real  scenes 
were  described.  What  remains  is  to  become  more  specific  about  why  these  properties  are 
computationally  useful. 

4.1  The  relation  between  a  surfaoe  contour  and  its  oontour  generator 

The  current  problem  is  to  determine  the  contour  generator  T  in  3-space  on  the  basis  of  its 
projection,  the  surface  contour  C.  To  make  this  problem  graphic,  observe  the  3-D  shape  of  the 
curve  in  figure  4.  We  are  asking  how  that  shape  may  be  determined. 

The  projection  will  be  restricted  to  be  orthographic.  This  restriction  would  hold  whenever  the 
dimensions  of  the  curve  in  space  are  small  relative  to  the  distance  from  the  curve  to  the  viewer. 
Orthographic  projection  is  linear,  hence  some  useful  geometrical  properties  are  preserved,  notably 
parallelism. 

Now,  in  determining  the  shape  of  contour  generators  in  3-space  we  are  confronted  with  a 
problem  wherever  the  tangent  to  the  contour  (its  slope)  is  discontinuous:  Is  that  discontinuity  the 
projection  of  a  discontinuity  in  tangent  along  the  contour  generator,  or  is  the  discontinuity  due  to 
the  adjoining  of  distinct  contour  generators  on  the  surface?  Since  this  cannot  be  answered  locally 
without  a  priori  knowledge  of  the  specific  surface,  we  follow  the  principle  of  least  commitment 
[Marr,  1976b]  and  partition  the  surface  contours  in  an  image  into  their  smooth  segments. 

4.1.1  Ceneral  position 

A  number  of  constraints  will  be  consequences  of  assuming  general  position  --  that  the  viewpoint  is 
such  that  images  from  nearby  viewpoints  would  not  present  significant  differences  in  the  geometry 
of  the  projected  contours.  By  this  we  rule  out  viewpoints  that  cause  accidental  alignments  which 
mislead.  For  instance,  if  a  contour  C  is  straight  from  viewpoint  V,  then  assuming  general  position, 
it  would  be  straight  from  a  similar  viewpoint  --  it  is  not  the  case  that  the  contour  generator  T  is 
curved  in  a  plane  but  that  plane  is  viewed  "edge  on"  so  that  the  image  of  T  is  foreshortened  into  a 
straight  line.  General  position  allows  one  to  infer  properties  of  contour  generators  on  the  basis  of 
their  images,  such  as  smoothness,  continuity,  and  parallelism. 

Our  first  application  of  general  position  is  as  follows.  Since  the  contour  C  is  smooth  and 
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continuous,  r  is  smooth  and  continuous.1  Furthermore,  in  general  position,  nearby  and  distinct 
points  on  T  project  to  nearby  and  distinct  point  on  C.  That  is,  there  are  no  kinks  or  loops  in  T 
hidden  by  the  particular  viewpoint.  In  short,  assuming  general  position  allows  us  to  consider  T  as 
a  smooth  wire  in  3-space.  Now  we  consider  additional  constraints  which  allow  us  to  determine  its 
shape. 

4.1.2  The  planarity  restriction 

If  the  contour  generator  T  is  constrained  to  be  planar,  the  shape  of  T  would  be  completely 
determined  by  the  equation  of  the  plane  containing  the  curve  given  its  orthographic  projection  C. 
Hence  the  planarity  restriction  reduces  the  problem  of  determining  T  to  that  of  finding  the  spatial 
orientation  of  the  plane  n  containing  T 

Since  the  contour  generator  T  is  determined  once  n  is  specified,  one  approach  is  to  impose  an  a 
priori  choice  of  n,  then  examine  the  shape  of  T  that  results.  That  is,  one  assumes  a  particular 
spatial  orientation  for  the  plane  containing  the  contour  generator.  But  there  do  not  appear  to  be 
any  reasonable  choices  for  n,  except  for  the  ground  plane,  i.e.,  the  horizontal  plane  defined  by 
gravity.  However  it  is  not  feasible  to  assume  that  all  surface  contours  are  projections  of  horizontal 
contour  generators. 

Alternatively,  one  may  make  a  priori  assumptions  about  the  shape  of  T  in  the  same  spirit  as 
assuming  that  T  is  planar.  Then  n  would  be  a  consequence  of  C  and  those  restrictions  on  I\ 
What  restrictions  can  be  reasonably  placed  on  T,  and  how  are  those  restrictions  to  be  phrased?  I 
shall  consider  two  --  symmetry  and  minimum  curvature  variation. 

4.1.3  Symmetry 

Bilateral  symmetry  is  commonly  found  in  nature  and  usually  preserved,  at  least  indirectly,  in 
orthographic  projection.  We  are  interested  in  symmetry,  for  evidence  of  symmetry  in  an  image  will 
provide  constraint  on  the  shape  of  T  We  start  with  the  usual  definition  of  a  bilaterally  symmetric, 
planar  curve  as  comprising  two  loci  of  points  that  are  reflections  of  each  other  across  a  straight  line, 
the  axis  of  symmetry  (figure  5a).  The  symmetric  points  are  equidistant  across  the  axis,  the  line 
connecting  any  two  symmetric  points  is  perpendicular  to  the  axis,  and  all  such  lines  are  therefore 


I.  We  would  like  to  say  something  about  the  smoothness  of  the  surface  directly  under  the  contour 
generator  on  the  basis  of  the  surface  contour  being  smooth,  but  unfortunately  that  does  not  follow  from 
general  position  as  stated.  The  smooth  contour  generator  may  lie  along  a  sharp  ridge,  for  instance. 
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Figure  5.  The  bilateral  symmetry  in  A  can  be  described  in  terms  of  correspondence  lines  which 
connect  symmetric  points  lying  equidistant  from  a  straight  line,  the  axis  of  symmetry.  The  parallel 
correspondence  lines  are  perpendicular  to  the  axis  of  symmetry.  In  B  the  correspondence  lines 
connecting  qualitatively  symmetric  segments  of  the  curve  are  also  parallel  but  make  an  oblique 
angle  0  with  the  axis  of  symmetry. 


I 
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parallel. 

In  any  orthographic  projection  of  thu  curve,  the  image  of  symmetric  points  are  equidistant 
across  the  image  of  the  axis,  the  correspondent  lines  connecting  those  points  are  parallel,  but  the 
correspondence  lines  are  no  longer  perpendicular  to  the  image  of  the  axis  in  general  (figure  bb). 
This  configuration  has  been  aptly  termed  “skewed  symmetry*  by  Kanade  and  Render  [19793.  If  a 
unique  line  can  be  found  that  behaves,  in  this  sense,  as  the  image  of  an  axis  of  symmetry,  then  by 
general  position  we  will  assume  that  the  planar  curve  in  space  is  bilaterally  symmetric.  That  is,  we 
have  criteria  for  detecting  bilateral  symmetry.  When  these  criteria  are  satisfied  in  an  image  we  may 
assume  that  it  is  not  coincidental,  that  it  would  also  be  satisfied  in  an  image  taken  from  a  different 
viewpoint  —  hence  due  to  actual  symmetry.  The  problem  that  remains  is  to  detect  the  images  of 
symmetric  pairs  of  points. 

Orthographic  projection  is  linear,  hence  a  number  of  properties  are  preserved  by  the 
transformation  including  midpoints,  points  of  inflection,  and  convexity  and  concavity  [Marr,  1976b], 
Marr  has  shown,  in  the  context  of  finding  the  axes  of  generalized  cones,  that  axial  symmetry  can  be 
efficiently  detected  by  the  qualitative  symmetry  between  convex  and  concave  segments,  rather  than 
on  a  point-by-point  basis.  This  extends  to  the  detection  of  bilateral  symmetry,  where  the 
correspondence  lines  between  qualitatively  symmetric  segments  would  be  parallel.  The  line  defined 
by  the  midpoints  of  the  correspondence  lines  would  be  the  image  of  the  axis  of  symmetry. 

Returning  to  the  problem  of  constraining  the  shape  of  the  contour  generator,  the  symmetry 
detected  in  C  constrains  T  to  be  symmetric  and  this  in  turn  constrains  the  orientation  of  the  plane 
n  containing  I\  Specifically,  II  must  be  oriented  relative  to  the  viewer  such  that,  given  C,  r  would 
be  symmetric  if  lying  on  n. 

This  constraint  is  simply  expressed  in  terms  of  the  correspondence  angle,  the  angle  in  the  image 
between  the  correspondence  line  and  the  projected  axis  of  symmetry  (figure  bb).  Since  the 
correspondence  angle  is  the  image  of  a  right  angle  on  the  surface,  the  magnitude  of  the 
correspondence  angle  0  constrains  the  possible  spatial  orientations  for  the  tangent  plane  at  that 
point  (see  figure  6). 

In  short,  T  is  presumed  symmetric  if  an  axis  of  symmetry  can  be  reconstructed  from  the 
midpoints  of  parallel  correspondence  lines,  where  the  correspondence  lines  are  constructed  between 
qualitatively  symmetric  segments  of  C.  The  correspondence  angle  then  constrains  the  spatial 
orientation  of  the  plane  containing  f. 
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4.1.4  Minimum  curvature  variation 

The  curvature  of  C  encodes  information  about  the  orientation  in  space  of  the  contour  generator  I\ 
if  T  is  planar  and  some  other  restrictions  hold.  Witkin  [1979]  has  shown  that  the  orientation  of  the 
plane  11  containing  T  may  be  estimated  on  the  basis  of  the  curvature  along  C  if  we  assume  that 
systematic  variations  in  the  curvature  that  resemble  foreshortening  are  due  to  foreshortening.  Then 
one  may  choose  that  plane  fl  that  maximally  accounts  for  the  variation  in  curvature  in  terms  of 
foreshortening.  The  following  assumptions  are  sufficient  to  allow  this  analysis: 

(a)  the  possible  surface  orientations  of  11  are  equally  likely, 

(b)  the  tangents  to  the  contour  generator  are  arbitrarily  aligned  relative  to 
the  viewer  (they  are  independent  of  slant  ir  and  tilt  r),  and 

(c)  the  curvature  along  the  contour  generator  is  independent  of  a,  r.  and 
the  orientation  relative  to  the  viewer  of  the  tangent  to  the  contour 
generator  T 

The  constraint  on  T  that  results  is  roughly  equivalent  to  assuming  that  the  variation  in  curvature 
along  F  is  minimum  [Witkin,  1979],  Then  the  variation  in  curvature  along  its  projection  C  may  be 
attributed  primarily  to  foreshortening,  whereupon  the  degree  of  foreshortening  --  hence  the 
orientation  of  the  plane  fl  containing  T  --  may  be  estirnated.  To  introduce  this,  consider  the  case 
when  r  is  a  circle,  a  planar  curve  with  constant  curvature.  The  orthographic  projection  C  is  an 
ellipse;  the  curvature  along  the  ellipse  varies  according  to  the  foreshortening  of  the  corresponding 
segment  of  the  circle.  One  may  derive  from  the  variance  in  curvature  an  estimate  of  the 
orientation  of  the  plane  containing  T. 

This  constraint  has  been  phrased  in  terms  of  minimum  curvature  variation,  but  Witkin 
describes  it  more  generally  as  a  problem  of  signal  detection.  The  "waveform"  that  we  consider  is 
the  contour  in  the  image  (parameterized  in  terms  of  contour  curvature).  The  curvature  at  any 
point  on  the  contour  consists  of  two  components,  one  being  the  curvature  of  the  contour  generator 
at  each  corresponding  point,  the  other  being  a  "projective  component’  which  increases  or  decreases 
the  apparent  curvature  according  to  the  orientation  of  the  given  segment  of  the  contour  generator 
relative  to  the  viewer  (in  the  circle  example,  where  the  tangent  lies  parallel  to  the  image  plane,  the 
curvature  on  the  ellipse  is  minimum;  where  the  tangent  to  the  circle  is  oriented  away  from  the 
viewer  the  curvature  is  greatest).  The  curvature  of  the  contour  generator  is  treated  as  noise;  the 
projective  component  is  the  signal.  Since  the  projection  is  orthographic  and  the  contour  generator 
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is  planar,  the  projective  component  will  be  regular. 

The  problem  of  determining  the  orientation  of  the  plane  containing  T  may  be  recast  as  that  of 
estimating  the  amplitude  and  phase  of  a  signal  of  known  waveform  (the  projective  component)  in 
the  presence  of  noise  (the  unknown  shape  of  T).  The  problem  can  then  be  solved  by  seeking  to 
account  for  as  much  as  possible  of  the  variance  in  the  surface  contour  in  terms  of  the  projective 
component.  The  constraint  stems  from  the  fact  that  the  processes  that  determine  the  shape  of 
contour  generators  on  actual  surfaces  usually  do  not  impose  the  same  kind  of  systematic  regularity 
as  that  imposed  by  orthographic  projection. 

4.2  The  relationship  between  a  contour  generator  and  the  surface 

Given  the  contour  generator  T  is  a  planar  3-D  curve,  how  does  the  surface  E  lie  under  T?  In  terms 
of  the  wire  and  ribbon,  a  primary  question  concerns  whether  the  ribbon  may  twist  along  the  wire. 
More  formally,  if  the  plane  containing  T  is  n,  does  the  angle  between  E  and  11  vary  along  T? 

A  result  in  differential  geometry  is  that  given  a  curve  T  defined  by  the  intersection  of  a  plane 
n  and  a  surface  E,  if  the  angle  between  E  and  n  is  constant  along  l",  T  is  a  line  of  curvature  (see, 
e.g.,  [O’Neill,  1966,  p.  224]).  Thus  if  the  contour  generator  is  planar,  and  that  plane  intersects  the 
surface  with  a  constant  angle,  the  contour  generator  is  a  line  of  curvature.  The  next  issue  is  to 
determine  the  angle  between  n  and  E. 

4.2.1  The  geodesic  and  asymptotic  restrictions 

If  the  plane  fl  containing  the  contour  generator  I*  is  perpendicular  to  E,  t  e  .  T  is  a  normal  section, 
then  T  is  geodesic.  Consequently  the  surface  normal  along  T  everywhere  coincides  with  the 
principal  normal  to  T  In  essence,  the  contour  generator  follows  a  path  on  the  surface  which  locally 
indicates  where  the  greatest  curvature  occurs.  The  binormal  to  the  contour  generator,  being 
perpendicular  to  both  the  principal  normal  and  the  tangent,  coincides  with  the  direction  of  least 
curvature.  However  all  such  binormals  are  parallel,  for  the  tangent  and  normal  along  I*  only  rotate 
in  the  plane  IT  Consequently  all  lines  of  least  curvature  are  parallel;  equivalently,  the  strip  of 
surface  under  the  contour  generator  is  a  cylinder. 

The  previous  discussion  considered  the  case  where  the  contour  generator  is  geodesic;  where  the 
angle  between  fl  and  E  is  ir/2.  If  that  angle  is  everywhere  zero,  then  n  coincides  with  the  tangent 
plane  of  E  and  the  surface  normal  along  I"  coincides  with  the  normal  to  FI.  As  mentioned  earlier  if 
a  curve  ties  in  a  plane  everywhere  tangent  to  the  surface  along  the  curve,  that  curve  is  asymptotic, 
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i.e.,  a  locus  of  points  of  zero  Gaussian  curvature.  The  importance  of  the  asymptotic  restriction  is 
found  in  gloss  contours.  The  contour  generators  corresponding  to  gloss  contours  in  the  image 
correspond  to  asymptotic  curves  on  the  surface.  Hence  where  gloss  contours  appear  we  know  that 
the  surface  is  locally  developable  (likewise,  where  point  specularities  occur  we  also  know  that  the 
surface  must  be  doubly  curved).  To  some  extent  we  may  further  understand  the  surface  geometry 
simply  on  the  basis  of  the  shape  of  the  contour  in  the  image  without  determining  the  particular  3-D 
shape  of  its  contour  generator.  If  the  contour  is  a  straight  line  in  the  image  we  cannot  tell  much, 
for  the  surface  may  be  either  cylindrical  or  twisting  (like  a  spiraling  piece  of  paper).  But  if  it  is  any 
smooth  curve  in  the  image  the  surface  is  roughly  planar  since  the  contour  generator  is  restricted  to 
be  planar  and  asymptotic. 

4.2.2  Parallelism  ^ 

The  discussion  thus  far  has  concerned  the  analysis  of  surface  shape  from  a  single  surface  contour. 
This  analysis  requires  that  the  contour  generator  T  may  be  determined  from  its  Image,  however  the 
constraint  afforded  by  planarity,  general  position,  symmetry,  and  constant  curvature  will  not  always 
allow  a  strong  determination  of  T  Ft  is  perhaps  not  coincidental  that,  in  fact,  our  perception  of 
surface  shape  from  a  single,  unfamiliar  contour  is  weak  when  compared  to  the  vivid  impression 
afforded  by  multiple,  parallel  contours  (figure  I).  The  basis  for  the  apparently  greater  constraint 
from  parallel  contours  will  now  be  discussed;  further  comments  on  the  psychology  of  surface 
contours  are  given  in  [Stevens,  1979]. 

If  surface  contours  are  parallel  in  the  image,  then  by  the  of  general  position,  their  contour 
generators  are  parallel.  The  fundamental  issue  now  concerns  the  behavior  of  the  surface  between 
the  contour  generators.  In  the  absence  of  independent  sources  of  information  about  the  surface 
such  as  shading  or  texture  we  must  make  some  a  priori  assumption  about  the  nature  of  the  surface 
between  the  contour  generators.  A  conservative  assumption  would  be  that  the  surface  extends  in  a 
"simple  manner"  between  them.  This  can  be  formalized  by  a  second  form  of  general  position:  that 
the  particular  positions  of  the  contour  generators  on  the  surface  are  not  critical,  that  if  shifted 
slightly,  the  contour  generators  would  project  qualitatively  the  same.  This  is  equivalent  to  assuming 
that  the  surface  is  a  cylinder  between  the  contour  generators. 

We  now  use  the  geodesic-asymptotic  restrictions  from  the  previous  section,  and  consider  two 
interpretations  for  the  cylindrical  surface:  Either  the  surface  is  (a)  curved  and  the  contour 
generators  are  parallel  geodesics,  or  (b)  flat  and  the  contour  generators  are  asymptotic  curves.  To 
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aid  in  visualizing  these  two  cases,  compare  figure  I  (geodesic  interpretation)  and  figure  7 
(asymptotic  interpretation).  These  effects  are  discussed  in  (Stevens,  1979].  Note  that  in  the  latter 
case  of  asymptotic  curves,  the  parallelism  does  not  provide  additional  constraint  on  the  surface 
solution  --  the  contour  generators  lie  in  the  same  plane.  Nor  does  the  shape  of  each  contour 
generator  in  the  plane;  it  is  as  if  the  curves  are  merely  arrayed  on  a  flat  surface.  The  interpretation 
of  parallel  contour  generators  as  geodesics,  however,  constrains  both  the  local  surface  orientation 
and  the  shape  of  the  contour  generators. 

4.2.3  Computing  parallel  correspondence 

Recall  that  the  angle  between  the  plane  containing  the  contour  generator  and  the  surface  is 
restricted  to  be  constant,  hence  the  contour  generator  is  a  line  of  (greatest)  curvature.  Also,  the  lines 
of  least  curvature  on  a  cylinder  are  straight,  parallel,  and  perpendicular  to  the  lines  greatest 
curvature.  If  line  flf  least  curvature  were  reconstructed  in  the  image,  the  angle  of  intersection  that 
it  would  make  wi^fi  a  surface  contour  (a  line  of  greatest  curvature)  would  be  the  projection  of  a 
right  angle.  This  angle  constrains  the  local  surface  orientation,  as  already  demonstrated  with 
regard  to  bilateral  symmetry.  In  fact,  the  lines  of  least  curvature  can  be  reconstructed. 

In  the  orthographic  image  of  a  cylinder  the  lines  of  least  curvature  would  project  as  straight 
and  parallel,  and  each  would  intersect  successive  surface  contours  at  a  constant  angle  (since  the 
contour  generators  are  parallel).  This  is  illustrated  in  figure  8.  Note  that  we  attempt  to  reconstruct 
only  the  projections  of  the  lines  of  least  curvature.  This  may  be  achieved  by  identifying  points  on 
adjacent  contours  whose  tangents  are  parallel  and  connecting  those  points  by  straight  lines  that  are 
parallel.  This  may  be  thought  of  as  bringing  points  on  adjacent  contours  into  parallel 
correspondence.  The  constructed  line  representing  the  image  of  a  line  of  least  curvature  will  be 
termed  a  correspondence  line.  Note  that  if  the  surface  contours  are  straight  for  a  portion  of  their 
length  (figure  9a)  the  tangent  to  a  point  P  on  one  contour  may  be  parallel  to  various  tangents  on 
the  adjacent  contour,  however  only  one  choice  would  result  in  a  correspondence  line  that  is  parallel 
to  the  other  correspondence  lines  between  curved  portions  of  adjacent  surface  contours  (figure  96)  ’ 

This  correspondence  is  unique  in  general,  and  therefore  may  be  used  as  a  constructive  criterion 


I.  Selection  of  that  choice  may  be  accomplished  by  a  local,  parallel  algorithm  similar  to  that  in  (Stevens, 
1978]. 
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Figure  8.  In  the  orthographic  image  of  a  cylindrical  surface  the  lines  of  least  curvature  project  as 
straight  and  parallel,  and  each  intersect  successive  surface  contours  at  a  constant  angle  Identifying 
points  on  adjacent  contours  whose  tangents  are  parallel  and  connecting  those  points  with  lines  that 
are  parallel  establishes  parallel  correspondence,  one  basis  for  postulating  that  the  underlying  surface 
is  a  cylinder  (subject  to  general  position). 
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Figure  10.  The  cylinder  restriction  is  only  local,  for  the  parallel  correspondence  need  only  be 
established  between  adjacent  surface  contours,  and  the  parallelism  between  reconstructed  lines  of 
least  curvature  is  defined  only  locally.  Consequently  the  local  cylinder  restriction  may  be  applied  to 
the  surface  contours  above  although  the  global  surface  is  not  cylindrical. 
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for  detecting  parallelism  between  surface  contours  and  for  postulating  that  the  surface  is  a  cylinder  * 

An  important  consequence  of  the  parallel  correspondence  is  that  the  surface  orientation  is 
necessarily  constant  along  the  lines  of  least  curvature  (in  orthographic  projection,  as  we  have  been 
assuming).  Thus  if  the  surface  orientation  were  determined  along  the  contour,  it  can  be  simply 
propagated  along  the  correspondence  lines  to  provide  a  complete,  interpolated  solution  to  the 
surface  orientation  across  the  cylindrical  surface  between  parallel  surface  contours. 

We  have  seen  that  assuming  that  the  contour  generator  T  is  planar  and  that  the  angle  between 
the  plane  containing  T  and  the  surface  is  constant  along  I”  restricts  the  surface  under  T  to  be  a 
cylinder.  Also,  for  parallel  surface  contours  the  two  forms  of  general  position  together  restict  the 
surface  to  be  a  cylinder.  Consequently,  the  curvature  of  the  surface  is  attributed  entirely  to  the 
curvature  of  the  contour  generator,  that  being  a  line  of  greatest  curvature. 

Note  that  the  cylinder  restriction  is  only  local,  for  the  parallel  correspondence  need  only  be 
established  between  adjacent  surface  contours,  and  the  parallelism  between  reconstructed  lines  of 
least  curvature  is  defined  only  locally.  Consequently,  the  cylinder  restriction  may  be  applied,  for 
example,  to  the  surface  contours  in  figure  10,  where  the  surface  may  be  approximated  locally  by 
patches  of  cylinders  while  the  global  surface  is  not  cylindrical. 

4.2.4  Opacity 

We  now  consider  the  constraint  afforded  by  restricting  the  surface  to  be  opaque.  In  general,  opacity 
does  not  significantly  restrict  the  shape  of  the  underlying  surface.  However  the  opacity  restriction  is 
important  if,  as  before,  the  contour  gen  ’rator  is  assumed  to  be  a  line  of  greatest  curvature  and  the 
surface  under  the  contour  generator  is  assumed  cylindrical.  In  the  following,  a  geometrical 
construction  will  be  described  that  shows  how  these  restrictions  constrain  the  range  of  orientations 
to  which  the  parallel  lines  of  least  curvature  would  project.  The  angle  between  those  lines  and  the 
tangent  to  the  surface  contour  is,  again,  the  projection  of  a  right  angle.  Thus  the  opacity  restriction 
is  useful  in  constraining  local  surface  orientation  in  the  same  manner  as  skewed  symmetry  and 
parallel  correspondence.  The  restriction  imposed  on  slant  and  tilt  as  a  function  of  this  angle  is 
shown  in  figure  6. 


I.  Note  that  the  correspondence  is  not  unique  if,  for  instance,  the  parallel  surface  contours  arc  periodic, 
as  in  figure  I.  One  solution  in  that  case  is  to  choose  the  parallel  solution  which  results  in  the  shortest 
correspondence  lines. 
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The  constraint  follows  from  the  fact  that  if  a  line  of  curvature  is  continuously  visible  from  a 
given  viewpoint,  so  must  an  adjacent  line  of  curvature.  This  can  be  described  geometrically  in  the 
following  way:  The  correspondence  lines  (the  projections  of  lines  of  least  curvature)  that  connect 
adjacent  surface  contours  would  make  no  intersections  with  the  surface  contours  except  at  their 
terminations.  That  is,  the  situation  in  figure  11a  would  be  disallowed.  (Note  that  in  figure  I,  where  > 

this  does  not  arise,  the  surface  may  be  transparent  nonetheless.)  Now,  given  a  single  surface 
contour  (the  image  of  a  line  of  greatest  curvature  on  a  cylinder)  we  have  some  constraint  on  where 
an  adjacent  line  of  curvature  would  project,  and  this  in  turn  constrains  the  local  surface  shape.  { 

This  constraint  is  conveniently  represented  by  the  Gauss  map  (see,  for  example,  [Hilbert  &  l 

Cohn-Vossen,  1952]).  A  Gauss  map  is  a  simple  representation  of  the  range  of  orientations  of  I 

tangents  along  a  curve.  The  given  curve  is  mapped  to  an  arc  on  a  unit  semi-circle  where  each  ^ 

point  on  the  curve  maps  to  the  point  on  the  semi-circle  whose  radius  is  parallel  to  the  tangent  to  f 

the  curve.  This  is  illustrated  in  figure  116.  Observe  how  tangents  at  various  points  P  map  to  | 

corresponding  points  on  the  semi-circle.  | 

The  next  step  is  to  use  the  Gauss  map  to  represent  the  range  of  possible  orientations  of  the 
correspondence  lines.  Let  that  orientation  be  a,  which  maps  to  a  single  point  P  on  the  semi  circle 
(that  point  P  whose  radius  has  the  orientation  a).  In  figure  12  three  choices  for  a  are  shown  which 
are  consistent  with  the  surface  being  opaque.  Now,  the  constraint  that  the  correspondence  lines  not 

i 

intersect  the  surface  contours  equates  to  the  restriction  that  the  point  P  (the  mapping  of  orientation 

a)  not  lie  on  the  arc  of  the  semi-circle  already  covered  by  the  surface  contour.  The  degree  of  j 

l 

constraint  imposed  by  the  opacity  restriction  depends  on  the  surface  contour.  In  figure  13a  the  j 

shallow  contour  maps  to  only  a  short  arc,  and  the  correspondence  lines  could  have  a  large  range  of 
orientations.  But  in  figure  136  the  correspondence  lines  are  restricted  to  a  narrow  range  of 
orientations. 

Given  that  the  correspondence  lines  are  the  projections  of  lines  of  least  curvature  which  on  a 
cylinder  are  identically  the  binormals  to  the  plane  containing  the  lines  of  greatest  curvature,  the 
orientation  to  which  the  correspondence  lines  projects  provides  us  with  the  tilt  component  of  surface 
orientation  for  the  plane  containing  the  given  curve. 

4.3  Criteria  governing  the  tangential/surfaoe  contour  decision 

Earlier  we  discussed  the  distinction  between  tangential  contours  (silhouette  boundaries  along  which 
the  line  of  sight  grazes  the  surface)  and  surface  contours,  noting  that  surface  contours  include 
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Figure  II.  The  opacity  restriction  disallows  the  correspondence  lines  (the  projections  of  lines  of  least 
curvature)  that  connect  adjacent  surface  contours  to  intersect  the  surface  contours  except  at  their 
terminations.  That  is,  the  situation  in  a  is  disallowed.  Opacity  provides  some  constraint  on  the 
relation  between  a  contour  generator  and  the  underlying  surface.  Towards  representing  this 
constraint,  we  represent  the  surface  contour  by  its  Gauss  map  onto  a  semi-circle,  as  in  b. 
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Figure  13.  The  image  of  the  lines  of  least  curvature  map  to  a  single  point  on  the  Gauss  map.  If 
opaque,  that  point  cannot  already  be  occupied  by  the  mapping  of  the  surface  contour.  In  a  the 
surface  contour  is  a  shallow  curve  which  maps  to  a  small  arc  on  the  Gauss  map.  This  does  not 
strongly  constrain  the  possible  orientations  of  the  correspondence  lines  (the  projected  lines  of  least 
curvature).  But  in  b  the  curve  covers  much  of  the  Gauss  map,  hence  the  orientation  of  the  lines  of 
least  curvature  is  strongly  constrained.  One  choice  of  that  orientation  is  shown,  and  the  position  of 
an  adjacent,  parallel  surface  contour  is  drawn.  The  opacity  restriction  then  provides  constraint  on 
surface  orientation  by  the  oblique  correspondence  angle. 
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silhouette  boundaries  that  are  not  tangential  contours.  Marr  [1976b]  has  delineated  properties  of 
the  silhouettes  of  generalized  cones  (whose  boundaries  are  tangential  contours)  —  surfaces  whose 
shape  can  be  recovered  from  their  silhouettes.  The  silhouette  of  a  generalized  cone  exhibits 
qualitative  symmetry;  where  the  correspondence  lines  connecting  symmetric  segments  of  the  contour 
would  be  perpendicular  to  the  axis  of  symmetry.  For  instance,  the  symmetric  silhouette  in  figure  2a 
is  generally  interpreted  as  a  vase-like  object,  and  the  contours  are  seen  as  tangential  contours. 

Similarly,  geometrical  criteria  can  be  given  which  indicate  that  a  contour  is  a  surface  contour. 
(Note  that  non-geometrical  means  also  exist,  e.g.,  determining  that  the  corresponding  contour 
generator  is  a  shadow  edge,  or  a  gloss  contour  or  a  discontinuity  in  surface  texture)  Two 
geometrical  criteria  are  suggested  by  the  preceding  discussion  First  consider  qualitative  symmetry 
where  the  correspondence  lines  are  not  perpendicular  to  the  axis  of  symmetry  (as  just  discussed  in 
the  case  of  bilateral  symmetry)  but  oblique  to  the  axis  (as  in  figure  56).  When  achieved,  this  skewed 
symmetry  suggests  a  surface  contour,  as  opposed  to  a  tangential  contour,  interpretation.  Secondly,  if 
parallel  correspondence  between  contours  can  be  achieved,  as  in  figure  36,  those  contours  can  be 
interpreted  as  surface  contours. 
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6.  SUMMARY 


1.  The  analysis  of  the  shape  of  a  surface  from  surface  contours  may  be  decomposed  into  two 
problems:  reconstructing  the  corresponding  3-D  curves  (the  contour  generators)  and  determining 
their  relation  to  the  surface.  This  decomposition  separates  the  problem  of  determining  the 
projective  geometry  from  that  of  determining  the  intrinsic  geometry. 

2.  The  first  problem  is  constrained  by  general  position,  planarity,  symmetry,  and  minimum 
curvature  variation. 

3.  The  second  problem  is  reduced  by  assuming  the  angle  between  the  surface  and  the  plane 
containing  the  contour  generator  is  constant.  Then  if  that  angle  is  a  right  angle,  the  contour 
generator  is  geodesic;  if  the  angle  is  lero,  the  contour  generator  is  asymptotic.  In  either  case  the 
contour  generator  is  also  a  line  of  curvature.  Since  it  is  also  planar,  the  surface  is  locally  a  cylinder. 

4.  We  also  arrived  at  the  cylinder  restriction  in  the  case  of  parallel  surface  contours,  given  the  two 
forms  of  the  principle  of  general  position.  The  opacity  restriction  is  also  useful,  given  the  planarity 
and  geodesic  restrictions,  in  understanding  how  the  surface  lies  under  a  contour  generator. 

5.  We  have  considered  instances  when  the  various  constraints  are  valid.  Surface  markings  on 
synthetic  and  biological  objects  and  the  edges  of  cast  shadows  are  often  geodesic  and  planar.  Gloss 
contours  are  asymptotic  and  planar,  at  least  in  the  case  of  distant  light  sources  and  orthographic 
projection.  Hence  if  the  contour  generator  can  be  reconstructed  as  a  curve  in  3-D,  the  surface 
orientation  along  the  curve  can  be  computed  subject  to  either  the  geodesic  or  asymptotic 
interpretations. 

6.  Constraints  on  the  intrinsic  geometry  are  also  provided  by  surface  contours  even  if  the  contour 
generator  is  not  well  determined  in  space:  Gloss  contours,  highlights,  and  shading  edges  tell  us  of 
the  local  Gaussian  curvature  in  some  cases. 
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PART  II:  TRANSPARENCY 
Whitman  Riohards  and  Andrew  P.  Witkin 

1.  INTRODUCTION 

1.1  The  problem 

The  intensity  of  light  reaching  a  point  in  an  image  such  as  the  retina  depends  upon  many 
parameters  of  the  scene.  These  include  the  strength  of  illumination,  reflective  properties  of  the 
viewed  surface,  and  the  orientation  of  the  surface  with  respect  to  the  viewer  and  the  Uluminant.  To 
interpret  an  image,  these  confounded  factors  must  be  sorted  out.  Important  steps  in  this  direction 
have  been  taken  by  Land  and  McCann  [1971]  and  by  Horn  [1975].  Land  and  McCann  have 
developed  a  method  for  differentiating  the  contributions  of  illumination  and  reflectance  for  simple 
scenes,  while  Horn  has  shown  how  to  isolate  the  contribution  of  surface  orientation. 

For  natural  scenes,  still  another  factor  may  confound  the  interpretation  of  image  intensities: 
when  a  partially  transparent  object  occludes  the  scene  behind,  then  the  resultant  image  is  a 
combination  of  the  properties  of  the  transparent  object  as  well  as  its  background.  How  can  the 
object  and  its  background  be  isolated  from  the  available  image  intensity  distributions?  Such 
situations  occur  frequently,  not  only  when  scenes  are  viewed  through  haze,  rain,  or  dirty  windows, 
but  also  when  a  tree  or  bush  is  interposed  between  the  object  of  interest  and  the  eye.  In  such 
situations,  the  image  content  on  the  retina  then  becomes  a  projection  of  both  the  transparent 
occluding  medium  and  the  scene  behind.  How  can  these  two  components  of  the  image  be  separated 
so  that  the  true  scene  content  can  be  deduced? 

1.2  Extending  the  definition  of  transparency 

When  viewed  at  a  distance,  a  dust  cloud  appears  to  be  a  homogeneous  partially  transparent 
medium.  But  if  we  get  close  enough  to  resolve  the  individual  dust  particles,  we  find  that  it  is 
composed  of  opaque  elements  suspended  in  a  completely  transparent  medium.  Does  this  mean  that 
the  cloud  is  no  longer  partially  transparent  beyond  the  point  at  which  its  structure  can  be  resolved? 
Clearly  any  collections  of  opaque  suspended  elements  --  i.e.,  "screens’  --  become  transparent  media 
whenever  the  elements  cannot  be  resolved.  The  distinction  between  a  "screen"  and  the  classical 
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transparent  "film"  is  merely  one  of  the  scale  of  resolution.  The  general  form  of  transparency, 
therefore,  includes  the  separation  of  such  "screens',  e.g.  trees  or  rain,  from  their  backgrounds.  The 
homogeneous  film  is  merely  a  special  limiting  case. 

A  general  definition  of  transparency  is  therefore  as  follows: 

A  surface  (or  medium)  will  be  said  to  be  transparent  if  the  projected 
image  content  depends  both  upon  the  medium  as  well  as  its  partially 
occluded  background. 

This  definition  allows  for  the  spatial  (or  temporal)  frequency  content  of  the  image  to  be  altered 
by  interposing  a  transparent  object  or  medium,  as  well  as  alterations  by  intensity,  color,  or  contrast 
changes.  Quite  simply,  a  transparent  medium  is  a  filter  that  passes  some  of  the  more  distant  scene 
content,  as  well  as  contributing  to  the  scene.  The  general  problem  is  to  assess  the  properties  of  this 
filter  and  its  background  from  image  information  alone. 

1.3  Historical:  When  oan  transparency  be  deduced  from  image 
information? 

Early  experimenters  readily  recognized  that  perceptual  transparency  does  not  correspond  to  physical 
transparency  [Metelli,  1970].  For  example,  air  is  physically  transparent,  but  we  do  not  see  air  as  a 
transparent  medium  unless  it  is  a  misty  day.  A  clear  pane  of  glass  is  another  example  of  a 
physically  transparent  medium  that  will  not  be  "seen"  as  transparent  unless  the  pane  has  a  dirty 
film  or  a  reflection  on  it.  Thus,  physical  transparency  is  not  a  necessary  condition  for  perceptual 
transparency.  Nor  is  it  sufficient,  for  consider  the  years  of  successful  experimentation  on 
transparency  using  opaque  cardboard  cut-outs  or  printed  illustrations  as  in  Figure  I  [Metelli,  1974]. 
Does  this  mean  therefore  that  physical  transparency  cannot  explain  perceptual  transparency? 

Although  air  (or  aqueous  humor)  is  physically  transparent,  and  not  seen,  should  this 
observation  exclude  us  from  considering  the  perception  of  transparency  as  an  attempt  to  determine 
whether  a  filter  has  been  interposed  between  retina  and  a  portion  of  the  external  scene?  Obviously 
a  clear  pane  of  glass  will  not  appear  transparent  because  no  information  about  the  presence  of  the 
glass  is  available  in  the  retinal  image.  Clearly,  if  transparent  media  are  to  be  "seen"  by  a  visual 
system,  then  information  about  transparency  must  be  available  in  the  retinal  image.  Understanding 
the  perception  of  transparency  must  therefore  begin  by  understanding  what  information  is 
potentially  available  in  the  image.  Such  an  analysis  is  a  first  step  toward  developing  a 
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computational  theory  of  transparency  (see  Marr  &  Poggio,  1977). 

2.  BEGINNING  A  COMPUTATIONAL  THEORY 

2.1  Measures  of  ''transparency” 

Although  the  basic  problem  is  to  deduce  and  describe  the  nature  of  a  transparent  medium  from 
image  information  alone,  a  first  step  is  to  determine  how  we  are  to  characterize  the  properties  of  the 
transparent  medium.  Relevant  descriptions  include  the  spatial  frequency  content,  the  color,  the 
reflectance,  and  the  transmittance  of  the  medium.  Our  analysis  will  be  restricted  to  untangling  the 
reflectance  and  transmittance  of  the  transparent  medium  from  the  reflectance  of  its  occluded 
background.  Spectral  information  will  also  be  considered,  but  in  a  subsequent  section. 

2.2  Relations  between  image  intensities,  refleotanoes  and 
transmittance 

If  no  texture  or  temporal  information  is  available  in  a  region  of  an  image,  then 
transparency  cannot  be  deduced  solely  from  image  intensities  lying  only  within  the  transparent 
region.  Under  these  conditions,  the  region  (such  as  P  in  Figure  1)  will  be  totally  homogeneous. 
Under  uniform  illumination,  we  can  designate  the  image  intensity  in  the  region  (P)  as  I,v,-Ip.  This 
average  intensity  image  will  be  determined  by  the  reflectivity  p,  of  the  transparent  medium  (P)  and 
its  transmittance,  r.  Because  some  light  from  the  background  B  also  contributes  to  the  observed 
image  intensity  in  region  P,  we  must  consider  its  intensity  IB,  which  will  be  the  product  of  the 
reflectance  of  the  background  pB  and  the  illumination  E  common  to  all  surfaces  in  the  scene.  The 
combined  intensities  Iiv|  seen  be  the  viewer  will  then  be 

Iiv|  -  E  ps  (1-D)  ♦  E  D -r-pB  (1) 

where  D  is  the  areal  density  of  the  transparent  regions  of  the  medium  and  (l-D)  is  the  density  of 
the  reflecting  particles  of  the  medium.  Thus,  if  the  medium  is  considered  as  a  screen,  the  fraction 
of  the  screen  that  reflects  light  is  (l-D),  whereas  D  is  the  fraction  of  the  screen  that  is  occupied  by 
"holes"  and  is  completely  transparent.  Of  course,  as  the  scale  of  the  screen  becomes  finer  and  finer, 
eventually  both  the  reflecting  particles  and  the  holes  between  them  become  invisible  to  the  observer, 
who  will  see  only  a  homogeneous  medium. 
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Returning  now  to  equation  (1),  the  problem  is  to  determine  the  reflectance  p$  and 
transmittance  r  of  the  medium  P  (which  may  be  a  fine  screen).  However,  in  region  P,  the  only 
information  available  in  the  projected  (retinal)  image  is  I,*,.  Clearly  we  cannot  recover  the 
reflectance  ps  and  transmittance  r  from  a  single  equation  of  one  known  (Itv|)  and  the  five 
unknowns  ps,  r,  E,t>  and  pB. 

The  next  obvious  option  is  to  consider  two  adjacent  regions,  such  as  P  and  B  in  figure  I.  If  it 
were  known  that  B  is  indeed  the  "background"  alone,  while  P  is  the  mixture  of  background  and 
an  interposed  screen  (or  some  other  medium),  then  the  image  intensity  IB  *  E-pB  is  known  for  this 
region  B  and  we  now  have: 


Iavg  -  E-P.0-D)  ♦  D-r-Ig  (2) 

where  the  first  term  on  the  R.H.S.  is  the  light  reflected  off  the  surface  of  P  and  the  second  term  is 
the  light  from  B  transmitted  through  P.  Since  1^  and  IB  are  now  known,  only  D,  pg,  E  and  r 
remain  as  unknowns.  However,  it  is  still  impossible  to  solve  for  ps  and  r  and  thus  the  properties 
of  P  still  cannot  be  separated  from  its  background  B.  Thus,  two  homogeneous  regions,  one  of 
which  is  known  to  be  the  background  still  do  not  provide  enough  information  to  determine 
whether  the  adjacent  region  (P)  is  transparent  or  opaque.  . 

If,  however,  the  areal  density  of  the  "holes"  in  P  can  be  estimated  independently  (such  as  by  a 
higher  resolution  process),  then  presumably  the  image  intensity  Is  -  p,-E  of  the  opaque  portion  of 
the  screen  can  also  be  measured  by  the  same  process.  Thus, 

IaVg  -  Vl-D)  ♦  D  r  IB  (3) 

and  since  D.  I,,  Itv|,  and  IB  are  available  from  the  image,  the  transmittance  of  the  region  P  can  be 
deduced  (but  not  its  reflectance!): 

«> 

In  many  cases,  higher  spatial  resolution  may  not  be  available  to  provide  independent  measures 
of  the  hole  density  D  and  the  intensity  I,  of  the  opaque  portion  of  the  screen.  (This  is  the  limiting 
condition  where  the  "screen"  is  seen  as  a  transparent  film  or  filter.)  Metelli  (1974)  has  noted  that  for 
this  condition,  two  independent  backgrounds  are  needed  to  assess  the  transmittance  of  the 
transparency.  As  in  figure  I,  let  A  be  the  second  background  whose  image  intensity  is  measured  as 
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IA  and  let  Q,be  the  region  of  the  transparent  medium  lying  over  A,  with  image  intensity  Iq.  Then 
if  P*Q,  is  indeed  a  physically  homogeneous  transparent  region,  both  D,  p,  and  r  will  be  constant 
over  P+Q.  We  thus  have  two  equations  with  four  unknowns,  E,  p(,  D  and  r: 


Ip  -  E‘Pg(l-D)  ♦  D-r-Ig 


Iq-  E'P,(1-D)  ♦  Dt’Ia 


(5) 


However,  by  treating  the  product  D-t  as  one  unknown,  we  see  that  the  remaining  unknowns  cancel 
to  yield: 

ip-lrt 

Dt-T-i — (6A) 
'B‘*A 

where  D-r-r  is  the  average  apparent  transmittance  of  the  medium  P*Q.  Note  that  as  before, 
extended  illumination  has  been  assumed  in  the  derivation  of  the  relation  (6A).  If  the  lower  surface 
B+A  is  diffuse  and  is  illuminated  through  the  transparency  P*Q,  then  the  true  average 
transmittance  is  the  square-root  of  the  R.H.S.  of  (6A),  since  in  that  case  the  light  reaching  the 
viewer  has  passed  through  the  transparent  medium  twice.  Considering  this  uncertainty  about  the 
iliuminant,  the  true  value  of  the  transmittance  r  cannot  be  determined.  Recognizing  however  that 
r  lies  between  0  and  I,  we  note  that  a  weaker  condition  for  P*Q, being  transparent  is 

«>7^-12’>0  <&B) 

-IB*  A 

Metelli  [1974]  proposes  that  whenever  equation  6B  is  satisfied,  then  P*Q,wil|  appear  transparent. 

In  summary,  if  high  spatial  resolution  is  not  available,  then  the  calculation  of  the  average 
transmittance  of  a  “region"  in  the  image  requires  the  inspection  of  four  mutually  adjacent  regions  in 
the  image.  This  is  the  minimum  configuration,  which  reduces  to  a  local  "x"  intersection  for 
homogeneous  fields.  In  this  case,  the  average  (relative)  reflectance  p,  of  the  screen  can  also  be 
deduced  and  is  found  to  be 


p,  -  p,  D 


(7) 


Note  that  only  relative  reflectance  of  the  transmitting  surface  can  be  found  because  the 
iluminant  strength  E  is  still  undetermined. 
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2.3  Apparent  transmittance,  r«  and  oontrast 

Equations  (4)  and  (6)  gave  two  different  values  for  transmittance,  depending  upon  whether  the  scale 
of  the  image  measurement  is  smaller  (equation  4)  or  much  larger  (equation  6)  than  the  hole  sue  of 
the  screen.  Clearly  where  the  image  scale  is  finer  than  the  hole  size,  then  the  transmittance  will 
equal  that  of  the  hole,  r,  whereas  when  the  image  grain  is  coarse,  then  the  observed  transmittance 
will  be  the  average  r.  What,  then,  is  the  appropriate  measure  for  the  intermediate  case? 

As  a  screen  becomes  finer  and  finer,  so  its  holes  cannot  be  resolved  (as  if _the  observer  had 
moved  away),  then  the  transmittance  goes  from  r  to  7,  with  the  latter  corresponding  to  the  case 
where  the  screen  becomes  a  homogeneous  transparent  film.  Clearly,  the  criterion  for  deciding 
whether  the  region  P+Q,  should  be  considered  a  screen  or  a  film  depends  upon  whether  the  "holes" 
are  visible  or  not.  Such  visibility  depends  upon  the  convolution  of  the  image  operator  with  the 
image  values,  but  can  be  simply  regarded  as  the  effective  contrast  of  the  step  across  the  edge  of  an 
opaque  element  to  the  "hole"  in  the  image.  Let  AH  be  the  true  intensity  change  across  this  border, 
and  AH*  be  the  image  value  (i.e.,  the  result  of  convolution  with  the  transfer  function  of  the 
imaging  process).  Then  the  apparent  transmittance  r*  can  be  defined  as 


r •>  -  7(1- 


AH*  - 
AH  t 


(8) 


Equation  (8)  can  be  rearranged  to  the  form: 

r#  ■  7  ♦  (l-D)f  (8) 

Because  of  resolution  losses  in  the  imaging  system,  AHo  <AH.  Therefore  as  the  contrast  between 
the  elements  and  "holes"  in  the  screen  goes  to  zero  either  by  resolution  losses  or  by  AH-0,  the 
apparent  transmittance  r«  will  approach  the  average  transmittance  r  described  by  equation  (6). 


2.4  Imposing  natural  constraints:  joints  and  "oraoks” 


Although  at  least  four  contiguous  regions  as  in  Figure  1  are  required  for  deducing  transparency 
from  homogeneous  fields  using  image  intensity  alone,  the  configuration  is  not  yet  minimal.  If  the 
two  backgrounds  A  and  B  are  different  structures  (as  opposed  to  region  A  being  merely  a  painted 
portion  of  B),  then  there  may  often  be  a  crack  or  joint  between  A  and  B  where  they  abut.  Of 
course,  this  "crack"  will  continue  into  the  region  P+Q,  creating  the  boundary  between  the 
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transparent  overlay.  The  crack  itself,  therefore,  is  a  third  intensity  profile  available  in  the  image, 
and  its  image  intensities  can  be  used  instead  of  region  A  (or  B). 

The  computational  advantage  of  such  "cracks"  when  they  appear  is  that,  usually,  almost  no 
light  escapes  them.  This  means  that  a  "crack"  seen  through  a  transparent  medium  gives 
approximately  the  pure  reflective  component  of  the  medium,  with  negligible  transmitted 
contribution  from  the  background.  That  is,  intensity  profiles  of  most  "cracks”  and  joints  approach 
a  contrast  step  of  100%  for  widths  that  are  small  in  relation  to  the  depth  of  the  crack  into  the 
surface.  Such  cracks  can  generally  be  identified  in  the  image  by  their  intensity  profiles.  Once 
isolated,  then  the  contrast  step  in  the  background  AC  can  be  taken  reliably  as  near  1.0.  This  useful 
natural  constraint  upon  images  may  be  exploited  if  the  transmittance  and  reflectance  relations  for 
the  transparent  medium  (equations  6  and  7)  are  recast  in  terms  of  contrast. 

Let  AC*  be  the  apparent  contrast  across  the  AB  border: 

ac««  -  «0A> 


and  AC*pq  be  the  corresponding  contrast  for  PQ: 


I 

AC*pq  -  y 


P+IQ, 


Substituting  (10A  and  B)  into  equation  (6)  we  find  the  average  "film”  transmittance  to  be 


(I0B) 


ib*ia 


Recasting  equations  10  in  terms  of  Iq/Ip  and  IA/IB  and  substituting  in  the  above: 


OIA) 


„  _  AC*P0  (1*&C*AB)  Ip 
r  AC*ab  (UAC*pq)  Ig 


(IIB) 


For  highly  transmitting  media,  such  as  "thin”  films,  haze,  smoke  or  rain,  equation  (IIA)  is  useful 
because  as  Ip-»I8,  the  transmittance  is  determined  by  the  contrast  reduction  alone.  In  particular,  for 
a  "crack",  AC*ab-I  and 
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r  -  2‘AC*po/<UAC<.po)  02) 

or  to  a  crude  first  approximation  (since  AC«pq  also  approaches  i)  the  transmittance  equals  the 
maximum  contrast  in  the  region  P+Q.  Thus,  for  a  textured  background  (that  include  “cracks” 
where  AC>:<ab-I)  and  a  homogeneous  transparency,  the  measured  distribution  of  contrasts  can  be 
used  to  estimate  transmittance.  Clearly  because  transparency  of  necessity  must  reduce  the  contrast 
in  the  scene,  the  distribution  of  gray  levels  must  become  narrower.  Equation  (12)  is  just  a  special 
instance  of  this  result. 

Equations  (10)  can  also  be  combined  with  equations  (6)  to  determine  the  relation  between  the 
average  reflectance  p  of  the  transparent  medium  and  apparent  contrast  in  region  P«Q 

Ppg  ■  (l-ACp<)Hp/E  (13) 

Note  again,  that  without  some  estimate  of  the  illuminant  strength  E,  the  absolute  reflectance  of  P+Q 
cannot  be  calculated  from  image  intensities.  However,  if  evidence  is  available  that  a  region  B  has 
maximum  reflectance  of  1,  then  since  IB-pB-E,  with  pB-l  the  reflectance  pp q  of  P*Q,  can  be 
estimated.  Two  properties  of  images  can  aid  in  this  estimate:  1)  the  spectral  reflectance  of  B  and 
the  maximum  intensity  adjacent  to  the  perimeter  of  P+Q.  provided  that  this  intensity  falls  upon  the 
expected  distribution  for  gray  levels. 


3.  TRANSLUCENCY 


In  a  still  more  general  case  of  achromatic  transparency,  the  screen  or  film  will  have  internal  scatter, 
creating  a  translucent  surface  if  the  holes  of  the  screen  are  not  visible  in  the  image.  Clearly  the 
scattered  light  will  reduce  contrast.  If  the  translucent  medium  lies  on  top  of  a  background  of 
reflectance  pB,  then  the  apparent  reflectance  of  the  translucent  surface  will  be 


- 


t2Pb 


1'Po’Pb 


04) 


where  r  is  the  internal  scatter  of  the  translucent  medium  and  p0  is  the  reflectance  of  the  medium 
with  a  completely  absorbing  background  [Kubelka-Munk,  1931,  1954].  The  image  intensity  Ip  will 
now  be 


Ip  ■  Ep0*Er2PB/(l-p0pB) 
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Figure  2.  Transparency  arrangements  used  in  the  present  experiments.  The  upper  configuration 
was  used  to  test  Metelli’s  proposal.  The  lower  configuration  illustrates  the  local  nature  of 
transparency  judgements.  As  long  as  distance  s  exceeds  about  one-half  degree,  the  panels  P  ♦  Q, 
and  P*  ♦  Q/  can  be  made  to  appear  transparent  although  the  intensity  relation  violate  the  rules  for 
physically  transparent  objects. 
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A  similar  equation  for  regions  Q,and  A  yield  the  following  solution  for  r2: 

2  Op  Iq)  «-PcPb>  O'PqPa) 

/i  i  w/.  (Ibp*  i*pb). 

('bU*  (d-Po-^i^T^y  >  (15) 

Although  Ip,  IB  and  I*  are  available  from  the  image,  p0,  pB  and  p*  are  not.  Hence  r2  cannot  be 
determined.  (However,  note  that  for  all  p’s,  the  quotient  always  satisfies  the  relation  0<r2<l  for  a 
transmitting  medium.)  In  the  special  case  where  po-0,  then  t2-(Ip-Iq/(Ib-Ia)  as  before. 

Now  consider  the  possibility  that  a  "crack"  or  edge  can  be  identified  in  the  image  (either  by 
occlusion  or  by  the  distribution  of  gray  scale  values  in  region  P).  Then  p*« 0  and  1**0,  simplifying 
(15)  to 

f 2  ■  -T- ^(I'PoPb)  (ISA) 

This  can  be  recast  in  terms  of  the  contrast  AC»pq.  For  small  ACopQ  values,  P»Q,and 

r 2  -  ACopQ  -j-^(l-p0pB)  (I6B) 

B 

An  especially  interesting  case  of  (16B)  is  when  pB~I  and  where  the  translucent  medium  is 
"white"  Then  the  absorption  is  trivial  and  r*(I-p)  and  Ip-Ig.  Substitution  in  equation  (16B) 
yields  # 

t**2AC*pq  (IOC) 

Thus,  the  apparent  translucence  yields  twice  the  image  contrast.  This  is  the  case  when  a  piece  of 
white  onion-skin  paper  lies  over  a  black  line  on  a  surface  (or  if  one  views  a  printed  sheet  from  the 
back  side).  Note  that  equation  (I6C)  is  the  same  as  equation  (12)  for  low  contrasts  between  P  and  Q. 
Films  of  low  transmittance  can  therefore  appear  equivalent  to  translucent  surfaces,  with  the 
transmittance  approximated  by  twice  the  maximum  image  contrast  provided  that  Ip  roughly  equals 


To  determine  an  upper  bound  on  the  validity  of  equation  (I6C),  consider  the  case  of  a 
translucent  surface  with  its  backing  removed.  Then  pB  may  approach  I  for  a  "white"  sheet  and 
extended  illumination  that  includes  back  illumination  of  the  translucency.  Letting  the  reflectance  of 
the  medium  be  p0  as  before,  Ip*p0'E  and  Ig-I-E,  and  equation  (I6C)  becomes 


r  ■  2  AC»pQ'Pe(I-pe). 


(16D) 
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(The  exponent  for  r  is  omitted  because  light  passes  through  only  once.)  Because  AC*pq<I  and 
because  0<pe(l-po)<25,  the  maximum  possible  value  of  r  is  0.5  if  pB  is  a  "white"  medium.  Equation 
(I6C)  thus  cannot  be  valid  if  it  yields  a  transmittance  greater  than  0.5.  A  conservative  position  for 
its  useful  upper  bound  will  be  to  make  inferences  from  equation  (I6C)  only  if  the  apparent 
transmittance  comes  to  less  than  0.25. 

4.  ACHROMATIC  SUMMARY 

Transparency  and  translucency  impose  different  relations  between  transmittance  and  image 
intensities,  even  if  the  illumination  conditions  are  the  same  and  known.  However,  for  both 
transparency  and  translucency  in  the  limiting  case  where  maximum  contrast  can  be  estimated  in  the 
region  overlaid  by  the  transparency  (or  translucency),  the  average  transmittance  of  a  homogeneous 
film  will  approximate  the  maximum  contrast  for  highly  transmitting  media.  Haze,  smoke,  rain  or 
mist  meet  these  conditions.  A  reasonable  image  measure  for  transmittance  is  thus  the  apparent 
contrast  reduction.  This  is  our  counterproposal  to  Metelli’s  theory  of  transparency.  The  contrast 
reduction  measure  offers  a  simple  approximation  to  the  exact  physical  solution.  It  holds  exactly  for 
highly  transmitting  media  and  offers  close  approximations  for  such  natural  transparencies  as  hare 
and  rain,  which  approach  those  limiting  conditions.  For  media  with  low  transmittances  (below  25£), 
twice  the  maximum  contrast  reduction  will  provide  a  crude  estimate  of  translucency  or 
transmittance. 

Unfortunately,  the  absolute  reflectance  of  the  transparent  (or  translucent)  medium  cannot  be 
easily  estimated  from  image  information  alone,  without  some  knowledge  of  the  illuminant  strength. 
If,  however,  the  reflectance  of  only  one  portion  of  the  adjacent  background  can  be  estimated 
[Richards,  1979],  then  the  reflectance  of  the  transparent  overlay  can  be  determined. 

5.  SPECTRAL  CONSIDERATIONS 

Under  achromatic  conditions,  a  most  reasonable  estimate  of  transmittance,  based  upon  image 
intensities  alone,  is  to  determine  the  maximum  contrast  and  set  this  equal  to  the  transmittance. 
This  determination  can  be  reinforced  by  evidence  obtained  from  "scratches"  or  cracks”  that  cross 
the  occluding  border.  Such  a  solution  is  based  upon  the  assumption  that  both  the  translucent  or 
transparent  media  as  well  as  their  backgrounds  have  flat  spectral  distributions  over  wavelengths  for 
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both  reflectance  and  transmittance.  In  the  natural  world,  however,  this  is  an  uncommon  occurrence. 
How  then  are  the  transmittance  and  reflectance  of  a  screening  surface  affected  when  spectral  factors 
are  considered? 

5.1  Why  predictions  based  upon  additive  mixtures  fail 

If  P+Q,  is  a  screen,  and  if  the  illuminant  is  extended,  with  a  flat  spectral  distribution,  then  the 
chromatic  content  of  (P*<£)  will  be  merely  the  mixture  sum  of  the  light  reflected  off  <P*Q}  plus  the 
light  transmitted  through  the  "holes"  coming  from  the  surfaces  B  and  A  below  (r  is  now  assumed 
to  have  a  flat  spectral  transmittance  over  wavelength).  If  the  colored  image  is  assessed  by  a 
trichromatic  system  similar  to  man’s,  then  the  color  content  of  P  and  Q,can  easily  be  deduced  from 
the  fact  that  all  mixtures  of  two  lights  might  be  on  straight  lines  in  a  chromaticity  diagram 
[Wysecki  and  Stiles,  1967], 

In  figure  3a,  let  the  x,y  (chromaticity)  coordinates  of  the  reflecting  surface  (P*Q)  be  at  point  p, 
locations  B  and  A  respectively  corresponding  to  the  two  backgrounds  (if  A-0,  then  its  location  is 
W.)  Then  all  mixtures  of  transmitted  and  reflected  light  will  be  on  the  two  loci  pB  and  p  A,  at  least 
for  the  case  where  the  "holes"  in  the  screen  have  an  flat  transmittance  function  over  wavelength. 
Thus  P  and  Qmust  lie  on  those  loci,  with  their  distance  from  p  proportional  to  D/G-D)  times  their 
tristimulus  sums  where  D  is  the  hole  density. 

The  outer  contour  in  the  CIE  diagram  denotes  the  spectral  limits  of  all  physically  realizable  x,y 
values.  Thus  p.  B,  and  A  must  all  lie  within  the  enclosed  boundary.  This  constraint  does  not 
preclude  the  possibility,  however,  that  P  and  Q.can  both  occupy  positions  on  the  opposite  side  of 
white  (W)  from  B  and  A.  For  example,  P  and  Q, might  be  greenish  blue  and  bluish  green,  while  B 
and  A  would  be  yellowish  green  and  orange.  Since  such  stimuli  are  seen  as  four  opaque  regions, 
rather  than  as  a  transparent  situation,  we  know  that  the  human  observer  does  not  consider 
homogeneous  transparent  surfaces  as  screens  that  obey  additive  color  mixture  laws. 

Perhaps  one  obvious  explanation  for  the  human's  failure  to  treat  transparencies  as  screens 
under  extended  illumination  is  that  such  an  interpretation  will  fail  seriously  if  the  illuminant  must 
first  pass  through  the  screen  to  reach  B  and  A  below.  In  this  case,  the  filtered  light  reaching  B 
and  A  will  have  a  strong  chromatic  component  from  P*Q,  which  with  even  minimal  diffusion  will 
seriously  alter  the  spectral  content  of  the  light  reflected  back  through  P*Q;  Because  the  spectral 
characteristics  of  P  and  B  (or  Q,and  A)  are  assumed  selective  and  different,  the  reflected  light  will 
be  reduced  depending  upon  the  extent  of  overlap  of  the  two  spectral  reflectance  functions  Ppq(X) 
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Figure  3.  Upper  figure  shows  the  standard  CIE  spectrum  locus  in  chromaticity  coordinates  x  and  y. 
According  to  classical  color  mixture  laws,  if  P  ♦  Q,  i*  a  transparent  overlay  above  regions  A  and  B. 
then  the  chromaticity  of  the  reflective  component  of  P  ♦  Q,  must  lie  at  p.  The  lower  figure  shows 
that  the  additive  mixture  rule  can  lead  to  imaginary  chroma ticities. 
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and  pB(X)  (or  pA(X)  for  A).  The  transmittance  of  P  (and  Q)  will  therefore  always  be  underestimated 
and  will  appear  to  be  zero  when  pp(X)  and  pB(X)  do  not  overlap  at  all,  for  example. 

The  same  argument  applies  even  more  forcefully  if  P*Q.  is  a  film  (either  transparent  or 
translucent).  In  this  case,  the  spectral  intensity  P(X)  in  the  image  will  always  be  less  than  the 
expected  mixture  sum  calculated  from  an  average  transmittance  r  and  reflectance  p. 

P(X)  -  E(X)r(X)pB(X)  ♦  E(X)  p(X)  <  IB(X)  r  ♦  E(X)p  (17) 

Depending  upon  the  overlap  in  the  relevant  spectral  distributions,  the  mixture  estimates  of 
transmittance  can  be  either  too  high  or  too  low.  For  example,  referring  to  equation  (6)  the 
difference  (IB-IA)  can  be  made  small,  but  (Ip-I<j)  can  be  made  very  large  by  letting  the  spectral 
reflectance  of  p  not  overlap  with  that  of  A,  but  with  considerable  overlap  with  pB  of  surface  B. 
Finally,  imaginary  reflectances  can  result.  Consider  figure  3b,  where  two  yellowish  surfaces  P«Q,are 
superimposed  upon  a  yellow-green  surface  B  and  an  orange  surface  A.  (As  shown  in  the  next 
section,  this  condition  arises  when  a  spectral  filter  of  no  reflectance  is  placed  over  A  and  B.) 
Observers  have  no  trouble  seeing  such  a  configuration  as  transparent,  yet  the  mixture  rule  predicts 
an  imaginary  spectral  reflectance  function  that  lies  outside  physically  realizable  surfaces  (Bolt,  et.  al, 
1977). 

Additive  color  mixture  rules  are  therefore  inappropriate  for  evaluating  the  transmittance  of 
screens  or  films,  where  subtactive  effects  must  occur.  Since  Metelli’s  scission  theory  is  essentially  an 
additive  mixture  theory  of  transparency,  it  fails  when  spectral  factors  are  considered. 

5.2  Subtractive  color  mixtures 

The  introduction  of  a  transparent  film  in  front  of  a  reflecting  surface  will  always  decrease  the 
spectral  bandwidth  of  the  transmitted  light,  thereby  increasing  its  purity.  The  farther  apart  the 
bandpass  characteristics  of  the  filter  r(X)  and  the  background  reflectance  pB(X),  the  greater  the 
increase  in  purity  of  the  transmitted  light  and  the  less  its  magnitude.  Under  the  common  condition 
where  the  chromaticities  of  r(X)  and  pB(X)  lie  on  complementary  sides  of  the  "white"  point,  then  the 
product  of  their  bandpassed  values  r^-pB(X)  will  approach  zero  as  purity  increases.  In  the  limiting 
case,  then,  the  reflectance  of  p ^  of  the  film  will  be  the  sole  determinant  of  the  spectral  content  of  P 
and  Q.  P  will  then  equal  Q,  and  P«Qmust  be  considered  opaque  because  no  hint  of  the  difference 
B-A  has  been  passed. 
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In  order  for  the  film  to  pass  some  c  *lity  of  B  and  A,  the  spectral  transmittance  of  the  film 
must  lie  near  or  within  the  triangle  WBA  (figure  4A).  (Quantitative  specifications  for  ideal  filters 
can  be  made,  relating  the  maximum  transmittance  possible  for  any  rypgfX)  combination.  The 
locus  for  this  product  exceeding  10*  is  not  too  distant  from  point  W.)  Now  consider  a  film  of  aero 
reflectance  and  spectral  transmittance  r^,  with  r\  lying  within  WBA.  Depending  upon  the 
strength  of  and  the  reflectances  p@  and  pA  for  surfaces  B  and  A,  the  imaged  chromaticities  of  B 
and  A  (i.e.,  P(X)  and  QjX))  must  move  from  location  r\  away  from  locations  B  and  A  to  new 
positions  B’-P  and  A’-Q.  But  the  greater  the  movement  toward  increasing  purity,  then  the  less  the 
net  light  transmitted.  Saturation  changes  will  thus  always  be  correlated  with  the  net  change  in 
image  intensity.  If  this  rule  is  violated,  the  region  P^Q, cannot  be  a  film. 

Note  that  in  this  case  where  the  film  has  no  reflectance  and  its  transmittance  lies  within  WBA, 
the  backgrounds  seen  through  the  film  must  become  more  saturated,  with  the  color  difference 
decreasing  (see  figure  4A).  This  is  just  the  configuration  that  leads  to  imaginary  reflectances  for 
additive  mixtures,  as  illustrated  in  figure  3B.  Thus,  if  imaginary  reflectances  are  predicted  by  the 
relative  positions  of  P,B  and  Q,A,  the  correct  interpretation  is  of  a  film  with  zero  reflectance,  with 
the  shift  in  chromatic  positions  being  due  solely  to  the  transmittance  of  the  film.  Civen  the  image 
intensities  of  P,  Q,  B  and  A,  together  with  their  chromaticities,  it  is  possible  (in  principle)  to  solve 
for  the  location  of  r  assuming  ideal  filters. 

Finally,  it  should  be  obvious  by  now  that  if  the  film  has  a  reflective  component  p^,  then  it  is 
not  generally  possible  to  find  unique  reflectances  and  transmittances  of  the  transparency  (Fig.  4B). 
Let  B(X)  be  the  spectral  reflectance  of  the  background,  p(X)  and  r(X)  be  the  transmittance  of  the 
film,  E(X)  the  illuminant  and  c( (X)  the  spectral  sensitivity  function  of  one  of  the  color  mechanisms 
(i-1,3)  then  the  light  transmitted  from  the  background  (assuming  extended  illumination)  and 
detected  by  the  i,h  color  mechanism  is 

Tjg  -  j B(X)-r(X)E(X)-Cj(X)dX 

Similarly,  the  strength  of  the  response  to  the  reflected  light  is: 


-/■ 


p(X)E(XR(X)dX 


In  region  P,  the  stimulus  to  the  i,h  color  mechanism  Pf  then  becomes: 


Pj  -  D  T|B*(I-D)Ri,  1-13 
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Figure  4.  The  effect  of  subtractive  color  mixture  upon  transparency  is  shown  in  the  upper  diagram 
(a).  The  interposition  of  a  filter  r,  must  change  the  chromaticities  of  the  backgrounds  A  and  B  to 
A1  and  B1  as  illustrated  by  the  arrows.  If  the  transparency  has  a  reflective  component,  p,  then 
additive  mixture  laws  apply  to  p,  B1,  and  A1  to  constrain  the  color  appearance  of  P  and  Q,  as 
shown  in  the  lower  diagram.  The  calculation  of  the  positions  of  r  and  p  solely  from  P,  Q.and  A,  B 
is  impossible  in  most  cases. 
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which  is  analogous  to  equation  (1),  where  D  is  the  hole  density.  For  two  backgrounds  A  and  B  and 
two  transparency  regions  P+Q,  we  now  have  twelve  stimulus  strengths  (scalar)  altogether.  The 
unknowns  are  five  functions  of  X:  E(X),  B(X),  A(X),  p(X),  r(X).  Unless  some  constraints  can  be 
imposed  upon  the  form  of  these  functions,  it  is  not  generally  possible  to  recover  their  values  from 
only  twelve  scalars.  (Beck’s  1978  claim  that  additive  and  subtractive  mixtures  yield  equivalent 
transparency  solutions  is  incorrect  because  it  ignores  both  X  and  the  illuminant.) 

6.3  Imposing  natural  constraints 

A.  "Crack"  constraint:  In  the  achromatic  case,  the  instance  where  region  A  was  a  "crack"  was 
helpful.  For  color  transparency,  the  "crack*  imposes  the  constraint  that 

Q4-(l*D)Rj 

because  the  transmitted  component  is  zero.  Thus 


Now  let  r0  be  the  achromatic  transmittance  (compute  by  taking  the  maximum  value  of  v(X)).  Then 
r(X)^r0.  and 

Ti^ro-IiB  “  roJ®(X)E(X)ci(X)dX 

and 


Pr<^  -  D  TiB  -  D^B(X)-r(X)-E(X)-ci(X)dX 


(19) 


Pj-Qi  -  DTib<D  r0I(B  1-1-3.  (20) 

where  IiB  is  the  image  intensity  for  region  B  seen  by  the  i,h  color  channel.  Since  D*r0  is  the 
average  maximum  transmittance  r0,  we  can  recast  equation  (20)  in  terms  of  chromatic  contrast 
similar  to  the  achromatic  use  (equation  12): 

r0^Tj  ■  2,AC<>iPQ/(l*AC«jpQ)  (21) 

where  AC**B  is  taken  as  1,  and  a  thin  "film”  is  assumed.  The  change  in  chromatic  contrast  in  any 
of  the  three  color  channels  in  the  region  of  the  "crack"  can  thus  not  exceed  the  maximum 
transmittance  of  the  film.  Note  that  this  rules  does  not  imply  that  the  change  in  chromatic  contrast 
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across  P  and  Qmust  always  be  less  than  the  change  across  B  and  A.  For  example,  if  B  is  greenish 
yellow  and  A  is  a  reddish  yellow  of  a  non-overlapping  spectral  function,  the  interposing  a 
blue-green  filter  can  cause  Q,(over  A)  to  become  black  while  leaving  P  (over  B)  light. 

B.  Saturation  Rule  for  Transparency:  By  extending  equations  (I0)-(12)  to  the  chromatic  case,  it  can 
be  shown  that  the  change  in  chromatic  contrast  (in  each  color  channel)  due  to  transparency  must  be 
less  than  the  change  in  contrast  in  the  background  (such  as  in  the  presence  of  a  "crack").  This  fact 
immediately  constrains  the  relations  between  P  and  Q.  in  the  (idealized)  chromaticity  diagram.  In 
Figure  5A,  the  location  of  the  blue  primary  /9  is  given  and  rays  are  drawn  to  background  regions  B 
and  A.  The  crosshatched  area  then  shows  the  maximum  change  in  the  blue  channel  (A/3)  possible 
in  the  transparent  region.  (Ideally,  logarithmic  coordinates  should  be  chosen.)  In  figure  5B,  the 
maximum  chromatic  contrast  for  the  red  channel  is  illustrated.  A  similar  rectangle  can  also  be 
constructed  for  the  green  channel.  Figure  5C  shows  the  size  of  the  limiting  polygon.  Its  position 
can  be  moved  within  an  approximate  chromaticity  diagram  based  on  logarithmic  coordinates. 

C.  Complementary  Color  Constraint:  Imagine  now  that  the  allowable  polygon  for  P*Q,  in  figure 
6A  moved  to  the  opposite  side  of  "white"  (W)  as  shown  by  the  dashed  outline  in  figure  6A.  The 
colors  are  then  complementary  to  those  of  the  background,  and  no  hint  of  the  background  colors  is 
apparent.  Although  such  a  transparent  medium  is  physically  possible,  transparency  cannot  be 
proven  from  image  information  alone.  The  more  conservative  procedure  would  thus  be  to  require 
that  transparent  medium  reveal  at  least  a  hint  of  the  background  colors  A  and  B.  This  constraint 
would  then  limit  the  range  of  P*Q.to  colors  lying  on  the  same  side  of  "white"  as  the  background,  as 
shown  by  the  various  positions  of  the  polygon  outline  in  figure  6B. 

6.  PRELIMINARY  PSYCHOPHYSICS 
6.1  Aohromatio  measurements 

Using  a  "Texture  Display"  driven  by  a  DEC  PDP-11  [Richards,  1978),  a  pattern  consisting  of  four 
regions  similar  to  figure  I  was  generated.  To  test  for  the  general  validity  of  equation  (6 A),  all 
possible  combinations  of  gray  level  arrangements  were  examined  for  transparency  using  four  gray 
levels:  .2,  .4,  .6,  and  .8  contrast.  (This  yielded  64  non-redundant  arrangements.)  Transparency  was 
seen  in  all  cases  where  equation  (6A)  was  satisfied  and  was  never  seen  when  equation  (6A)  was  not 
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Figure  5.  Saturation  rule  for  transparency  is  Illustrated  in  the  upper  diagram.  The  blue  primary  is 
located  at  the  intersection  of  the  dashed  lines.  The  cross-hatched  region  shows  schematically  the 
maximum  allowable  change  in  the  relativt  value  of  this  primary  due  to  interposing  a  transparency 
that  acts  to  degrade  contrast.  In  the  lower  diagram,  maximum  allowable  change  for  the  relative 
strength  of  the  red  primary  is  illustrated. 
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Figure  6.  By  combining  the  saturation  constraints  illustrated  in  Fig.  5,  a  polygon  about  the 
background  chromaticities  A  and  B  can  be  constructed.  The  net  result  is  illustrated  in  the  upper 
diagram.  In  an  ideal  color  coordinate  system,  the  displacement  of  this  polygon  to  any  region  sets  a 
limitation  on  the  distance  between  P  and  Q,in  any  transparent  overlay.  In  the  lower  diagram  the 
possible  positions  of  the  polygon  are  limited  to  the  same  side  of  the  white  point,  w,  as  the 
background  A  and  B.  This  restriction  is  imposed  by  the  rule  that  some  hint  of  the  color  of  both  A 
and  B  must  appear  in  P  and  Q,  excluding  P  and  (Rvalues  that  are  complementary  to  A  and  B. 
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satisfied. 

Such  measurements,  however,  do  not  test  the  hypothesis  that  contrast  is  used  to  assess 
transparency,  because  no  subjective  estimates  of  the  transmittance  were  required.  As  a  preliminary 
test  for  equation  (11)  and  (16)  thin  neutral  density  (W ratten)  filters  were  placed  over  four  pairs  of 
gray-white  backgrounds  of  reflectance  ranging  from  .07  to  .90  with  a  contrast  step  of  approximately 
0.2.  Figure  7  shows  the  transmittance  estimates  for  two  observers  averaged  over  the  four 
background  conditions.  Note  that  estimated  transmittance  for  most  of  the  range  follows  a  power 
relation  with  an  exponent  of  about  1/1.5.  This  solution  is  intermediate  to  the  case  where  the 
illumination  is  extended  (predicted  slope  is  1.0)  and  where  the  illuminant  must  pass  through  the 
filter  twice--once  to  illuminate  the  background  and  once  again  to  be  seen  by  the  viewer  (predicted 
slope  is  1/2).  The  human  observer’s  estimate  is  thus  a  compromise  between  these  two  common  cases; 
the  result  is  most  likely  due  simply  to  the  compressive  transformation  of  image  intensity  into  a 
lightness  scale.  In  other  words,  if  transmittance  is  estimated  using  apparent  contrast,  a  result  like 
Fig.  7  would  be  expected. 

0.2  Chromatic  measurements 

Using  an  Apple  II  computer,  various  combinations  of  colors  were  painted  in  the  regions  P,  Q,  B, 
and  A  of  figure  I.  Although  only  15  colors  were  available  for  any  one  setting  of  the  TV  color 
control  knob,  by  adjusting  the  tints  over  the  full  range,  a  wide  assortment  of  color  combinations 
could  be  examined.  The  principal  findings  were; 

1)  Transparency  was  never  seen  if  the  overlay  (P*Q)  did  not  contain  at 
least  some  hint  of  the  background  colors  (B*A).  (This  is  the 
complementary  color  constraint  introduced  in  section  5.3c.) 

2)  The  overlay  (P»Q)  also  was  required  to  have  a  common  color  for 
transparency  to  be  seen. 

3)  The  intensity  relations  should  roughly  (but  not  exactly)  satisfy  equation 
(6A). 

These  preliminary  findings  regarding  the  spectral  factors  used  by  man  in  assessing  transparency 
verify  the  major  set  of  constraints  upon  the  positions  of  P+Qin  the  chromaticity  diagrams  outlined 
in  the  earlier  sections.  However,  there  appears  to  be  a  considerable  relaxation  in  the  allowable 
positions  of  P  and  Q,  in  the  rectangular  region  defined  by  B,  A,  and  W  for  P«Q,  (see  figure  8). 
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ACTUAL  TRANSMITTANCE 


Figure  7.  Transmittance  estimates  for  two  observers  (WR,  AW)  for  this  films  having  essentially  no 
reflective  component.  The  data  fall  intermediate  between  the  assumption  of  extended  illumination 
(lower  dashed  line)  and  the  illumination  of  the  background  by  light  that  must  first  pass  through 
the  filter  (upper  dashed  line).  In  the  actual  experiment,  the  illumination  was  extended. 
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Figure  8.  The  cross-hatched  region  shows  the  empirical  observation  of  the  range  of  allowable 
chromaticities  for  the  appearance  of  transparency  given  the  two  background  chromaticities  A  and 
B.  To  appear  transparent,  the  overlay  must  reveal  some  hint  of  the  underlying  background  colors. 
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Perhaps  by  a  closer  scrutiny  of  the  relations  between  the  achromatic  and  chromatic  factors  in  both 
transparency  and  translucency,  the  reason  for  the  relaxation  of  the  constraints  can  be  determined. 

6.3  Dependence  of  peroeived  transparency  on  looal  processes 

The  psychophysical  investigation  of  perceptual  transparency  revealed  an  unexpected  local 
component  in  the  human  observer's  transparency  computation.  As  discussed  earlier  in  section  2.2, 
two  regions  P  and  B  do  not  determine  a  unique  transparency  solution.  Four  regions  P.QA.B  are 
sufficient  to  determine  either  a  unique  transmittance  and  relative  reflectance,  if  a  solution  exists,  or 
to  determine  that  P*Q, cannot  be  a  homogeneous  transparent  medium.  If  two  additional  regions,  P* 
and  are  added,  the  equations  are  over-determined.  It  is  therefore  possible  to  arrange  the 
intensities  of  the  six  regions  so  that  any  combination  of  four  (P-Q;A-B,  P’-Q|-A-B,  or  A-Q,  Q,’-B) 
determine  a  unique  solution,  but  all  the  regions  taken  together  are  inconsistent. 

Surprisingly,  the  human  observer  is  entirely  insensitive  to  such  inconsistency,  unless  the 
inconsistency  can  be  discovered  by  examining  some  single  region  of  the  image  subtending  less  than 
about  one  degree.  For  example,  in  Fig.  2B  set  the  image  intensities  of  P-P'  but  let  Q,and  q)  differ. 
Then  A  cannot  be  physically  transparent,  yet  it  will  still  appear  as  a  homogeneous  transparent  filter 
unless  local  evidence  of  the  inconsistency  is  available.  Only  the  spatial  relations  separating  P  and 
P’  need  to  be  changed  to  alter  the  perception  of  transparency.  When  S  is  large,  then  local 
simultaneous  comparison  of  all  six  regions  is  not  possible,  and  transparency  is  perceived.  When  the 
regions  are  moved  closer,  the  inconsistency  is  detected,  and  perceptual  transparency  is  lost.  This 
effect  demonstrates  that  perceived  transparency  depends  at  least  in  part  on  a  process  of  local 
comparison  of  intensities. 


7.  SUMMARY 

With  uncertainty  in  the  exact  illuminant  conditions,  the  most  useful  cue  for  assessing  the 
transparency  of  a  medium  is  contrast  reduction--either  achromatic  or  chromatic.  For  many 
instances  of  transparency  or  translucency  (haze,  fog,  or  rain)  the  maximum  contrast  In  the  scene  is  a 
useful  measure  of  the  transmittance. 

Spectral  information  can  provide  further  support  for  achromatic  deductions  of  transparency. 
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Man  appears  to  use  spectra)  information  conservatively,  however,  and  his  rules  exclude  many 
instances  where  transparency  is  possible.  The  principal  rules  for  transparency  are  that  1)  a  portion 
of  the  background  color  must  "leak  through"  and  2)  the  transparent  medium  must  possess  at  least  a 
hint  of  the  one  color  throughout  the  region  of  the  transparent  overlay.  These  rules  favor  the 
assessment  of  "whitish"  transparent  media  such  as  hate,  fog  or  rain. 

Acknowledgements:  Dr.  C.  Haddad  and  Major  J.  Thorpe  provided  helpful  criticism  on  drafts  of 
this  report.  The  technical  assistance  of  C.  Papineau  was  also  greatly  appreciated. 


Richards  &  Stevens 


-72- 


References 


REFERENCES 

Beck,  J.  1978  Additive  and  subtractive  mixture  in  color  transparency.  Percept,  id  Psychophysics 
23,  265-267. 

Bolt.  R.  A.,  Negroponte,  N.  and  Tom,  V.  1977  Color  transparency  effects  from  mosaics  on  opaque 
color.  Final  Report,  V.  S.  Army  Contract  *DAAG29-76-0037.  M.l.T.  Architecture  Machine  Croup. 

Gibson,  J.J.  1950  The  perception  of  the  visual  world.  Boston:  Houghton  Mifflin. 

Hilbert  D.,  Sc  Cohn-Vossen,  S.  1952  Geometry  and  the  Imagination.  Chelsea  Publishing. 

Horn,  B.K.P.  1975  Obtaining  shape  from  shading  information.  In  The  psychology  of  computer  vision, 
P.  H.  Winston,  ed.  New  York:  McGraw-Hill. 

Kanade,  T.,  and  Render,  J.R.  1979  Skewed  symmetry:  Mapping  image  regularities  into  shape. 
Technical  Report,  Computer  Science  Department,  Carnegie-Mellon  University,  (forthcoming). 

Koffka,  K.  1935  Principles  of  Gestalt  Psychology.  New  York:  Harcourt,  Brace  and  World. 

Kennedy,  J.M.  1974  A  psychology  of  picture  perception.  San  Francisco:  Jossey-Bass. 

Rubella,  P.  8c  Munk,  F.  1934  Ein  Bertiag  zur  Optik  des  Farbanstriche.  Z.  techn.  Physik  12.  593. 

Rubella,  P.  1954  New  contributions  to  the  optics  of  intensely  light-scattering  materials,  Part  11. 
Non-homogeneous  layers,  f.  Opt.  Soc.  Am.  44,  330. 

Land.  E.  H.  Sc  McCann.  J.  J.  1971  Lightness  and  retinex  theory,  f.  Opt.  Soc.  Am.  6!,  Ml. 

Marr,  D.  1976  Artificial  Intelligence  -  a  personal  view.  Artificial  Intelligence  9,  37-48.  Also  available 
as  M.l.T.  A.!.  Memo  355. 

Marr,  D.  1976  Early  processsing  of  visual  information.  Phil.  Trans.  Roy.  Soc.  B.  275,  483-524. 

Marr,  D.  1976  Analysis  of  occluding  contour.  Proc.  R.  Soc.  bond.  B.  197,  441-475.  Also  available  as 
M.l.T.  A.I.  Lab.  Memo  372. 

Marr,  D.  8c  Poggko,  T.  1977  From  understanding  computation  to  understanding  neural  circuitry. 
Neurosci.  Res.  Prog.  Bull.  15  ,  470-488. 

Metelli,  F.  1970  An  algebraic  development  of  the  theory  of  transparency.  Ergonomics  13  (1),  59-66. 

Metelli,  F.  1974  The  perception  of  transparency.  Set.  Am.  230  (4),  125-135. 

Postman,  L.  Sc  Tolman,  E.C.  1959  Brunswick's  Probabilistic  Functionalism.  In  Psychology:  A  study  of 
a  science.  Edited  by  S.  Koch.  New  York:  McGraw-Hill. 

Richards,  W.  1979  Why  rods  and  cones?  Biol.  Cyber.  33,  125-135. 


i 


Richards  &  Stevens 


-73- 


References 


Richards,  W.  1978  Experiments  in  texture  perception.  Final  Report,  AFOSR  Contract 
F44620-74-C-0076,  pp.  1-116.  (Available  from  M.I.T.  Psychology  Department,  Cambridge,  Mass. 
02139.) 

Stevens,  K.A.  1978  Computation  of  locally  parallel  structure.  Biological  Cybernetics  29, 19-28.  Also 
available  as  M.I.T.  A.I.  Lab  Memo  392. 

Rock,  I.  1975  An  Introduction  to  Perception.  MacMiUan:N.Y. 

Stevens,  K.A.  1979  Surface  perception  from  local  analysis  of  texture  and  contour.  M.I.T.  A.I.  Lab 
TR-512. 

Waltz,  D.  1975  Undersanding  line  drawings  of  scenes  with  shadows.  In  The  Psychology  of  computer 
vision,  P.H.  Winston,  ed.  New  York:  McGraw-Hill. 

Witkin,  A.  1979  Shape  from  contour.  M.I.T.  Ph.D.  Thesis  (Psychology),  forthcoming. 

Wysecki,  G.  Sc  Stiles,  W.  S.  1987  Color  Science.  New  York:  Wiley. 


